2024-04-25T16:52:52.625 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T16:52:52.625 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} --owner scheduled_pdonnell@teuthology --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283 --verbose -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/orig.config.yaml 2024-04-25T16:52:52.684 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T16:52:52.776 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673283' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF9z64+QXSgcdLDWBPIYf/YDXzKCo4Bi5inpv83swoO/NmpxnYYXt4ID6bTsCWRmpHrilj0LGgZx2LWx3MTeMhs= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOoVkFT7nDvvGZXUSIWYUe8BtoTt4sZbCEp0+4cYQu52haOHR+uUsgQLkrHl8Xai+Vi0BBBUXsTncbBkXiJm8Ak= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEfJRahg+D3SKayGUYXBVRJPB4ALo1qK6f8bQxJwztvSIQob633uMQuwWiIgdqMXXM2HEwPOeyCIP2+ChvTS1R0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T16:52:52.776 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T16:52:52.777 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T16:52:52.777 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T16:52:52.779 INFO:teuthology.task.internal:Checking packages... 2024-04-25T16:52:52.802 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T16:52:52.803 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T16:52:52.803 INFO:teuthology.packaging:ref: None 2024-04-25T16:52:52.803 INFO:teuthology.packaging:tag: None 2024-04-25T16:52:52.803 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:52:52.803 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:52:52.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:52:53.091 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T16:52:53.093 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T16:52:53.404 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T16:52:53.404 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T16:52:53.463 INFO:teuthology.task.internal:Saving configuration 2024-04-25T16:52:53.489 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T16:52:53.498 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T16:52:53.520 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:47:35.167934', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF9z64+QXSgcdLDWBPIYf/YDXzKCo4Bi5inpv83swoO/NmpxnYYXt4ID6bTsCWRmpHrilj0LGgZx2LWx3MTeMhs='} 2024-04-25T16:52:53.543 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:47:35.169458', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOoVkFT7nDvvGZXUSIWYUe8BtoTt4sZbCEp0+4cYQu52haOHR+uUsgQLkrHl8Xai+Vi0BBBUXsTncbBkXiJm8Ak='} 2024-04-25T16:52:53.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:47:35.170885', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEfJRahg+D3SKayGUYXBVRJPB4ALo1qK6f8bQxJwztvSIQob633uMQuwWiIgdqMXXM2HEwPOeyCIP2+ChvTS1R0='} 2024-04-25T16:52:53.564 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T16:52:53.570 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T16:52:53.570 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T16:52:53.570 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T16:52:53.570 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T16:52:53.673 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feeb16240d0>, signals=[15]) 2024-04-25T16:52:53.673 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T16:52:53.679 INFO:teuthology.task.internal:Opening connections... 2024-04-25T16:52:53.680 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T16:52:53.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:52:53.758 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T16:52:53.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:52:53.834 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T16:52:53.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:52:53.910 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T16:52:53.916 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-04-25T16:52:53.934 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-04-25T16:52:53.935 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:52:53.992 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-04-25T16:52:53.993 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:52:53.993 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:52:53.993 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:52:53.993 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:52:53.993 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:52:53.993 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:52:53.994 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-04-25T16:52:54.105 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-25T16:52:54.122 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-25T16:52:54.123 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-25T16:52:54.180 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-04-25T16:52:54.180 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-04-25T16:52:54.180 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-04-25T16:52:54.180 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:52:54.180 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:52:54.181 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:52:54.181 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-25T16:52:54.202 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-04-25T16:52:54.220 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-04-25T16:52:54.221 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-04-25T16:52:54.279 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:52:54.280 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:52:54.280 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:52:54.280 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:52:54.280 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:52:54.280 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:52:54.280 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-04-25T16:52:54.311 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T16:52:54.321 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T16:52:54.337 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T16:52:54.337 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:52:54.340 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:52:54.342 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:52:54.358 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T16:52:54.364 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T16:52:54.364 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:52:54.397 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:52:54.399 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:52:54.495 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T16:52:54.571 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T16:52:54.585 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T16:52:54.586 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T16:52:54.586 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T16:52:54.586 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:52:54.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:52:54.603 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-04-25T16:52:54.661 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-437.el9.x86_64 2024-04-25T16:52:54.661 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-437.el9.x86_64 2024-04-25T16:52:54.661 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-04-25T16:52:57.056 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T16:52:59.073 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T16:53:03.274 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T16:53:04.647 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 16 kB 00:00 2024-04-25T16:53:06.407 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T16:53:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-25T16:53:13.362 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T16:53:13.362 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T16:53:13.462 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T16:53:13.463 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-25T16:53:13.463 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T16:53:13.575 DEBUG:teuthology.orchestra.run.smithi096:> echo no | sudo yum reinstall kernel || true 2024-04-25T16:53:14.073 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 04:53:11 PM UTC. 2024-04-25T16:53:14.230 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 255 k 2024-04-25T16:53:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-04-25T16:53:14.243 INFO:teuthology.orchestra.run.smithi096.stderr:Operation aborted. 2024-04-25T16:53:14.301 INFO:teuthology.orchestra.run.smithi096.stdout:Is this ok [y/N]: 2024-04-25T16:53:14.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum reinstall -y kernel || true 2024-04-25T16:53:14.790 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 04:53:11 PM UTC. 2024-04-25T16:53:14.907 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 255 k 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-04-25T16:53:14.908 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-25T16:53:15.316 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.9 MB/s | 255 kB 00:00 2024-04-25T16:53:15.316 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:53:15.316 INFO:teuthology.orchestra.run.smithi096.stdout:Total 624 kB/s | 255 kB 00:00 2024-04-25T16:53:15.337 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-25T16:53:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-25T16:53:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-25T16:53:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-25T16:53:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-25T16:53:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-25T16:53:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:53:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:53:16.500 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:53:16.500 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:53:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:53:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:53:17.188 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalled: 2024-04-25T16:53:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T16:53:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:53:17.189 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T16:53:17.353 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T16:53:17.399 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T16:53:17.400 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T16:53:17.400 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T16:53:17.400 DEBUG:teuthology.parallel:result is None 2024-04-25T16:53:17.400 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T16:53:17.407 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T16:53:17.408 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:53:17.410 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:53:17.412 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:53:17.429 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T16:53:17.438 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T16:53:17.447 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T16:53:17.447 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:53:17.464 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:53:17.469 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:53:17.585 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T16:53:17.591 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T16:53:17.591 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:53:17.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:53:17.611 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:53:17.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:53:17.628 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:53:17.631 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:53:17.654 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:53:17.669 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:53:17.690 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:53:17.703 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:53:17.710 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:53:17.714 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:53:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:53:17.749 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T16:53:17.756 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T16:53:17.757 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:53:17.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:53:17.761 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:53:17.787 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T16:53:17.795 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T16:53:17.795 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:53:17.822 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:53:17.829 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:53:17.845 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:53:17.911 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:53:17.963 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:53:18.046 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:53:18.081 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T16:53:18.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:53:18.144 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:53:18.189 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:53:18.254 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:53:18.297 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:53:18.338 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T16:53:18.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:53:18.403 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:53:18.449 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:53:18.503 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:53:18.587 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:53:18.629 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T16:53:18.629 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:53:18.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-04-25T16:53:18.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-04-25T16:53:18.700 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-04-25T16:53:18.733 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:53:18.748 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:53:18.773 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:53:19.228 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T16:53:19.236 INFO:teuthology.task.internal:Starting timer... 2024-04-25T16:53:19.237 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T16:53:19.246 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T16:53:19.258 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T16:53:19.259 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-04-25T16:53:19.314 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-04-25T16:53:19.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-04-25T16:53:19.388 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-04-25T16:53:19.390 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-04-25T16:53:19.460 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-04-25T16:53:19.462 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T16:53:19.462 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-04-25T16:53:19.490 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-04-25T16:53:19.490 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-04-25T16:53:19.521 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-04-25T16:53:19.522 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-04-25T16:53:19.547 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-04-25T16:53:19.548 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-04-25T16:53:19.548 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:53:19.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:53:19.573 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:53:19.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:53:19.599 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:53:19.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:53:19.625 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T16:53:19.626 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:53:19.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:53:19.679 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:53:19.704 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T16:53:19.713 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T16:53:19.718 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T16:53:19.840 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T16:53:19.855 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T16:53:19.856 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T16:57:16.856 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-04-25T16:57:16.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-25T16:57:16.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:57:16.935 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-25T16:57:17.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-25T16:57:17.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T16:57:17.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:57:17.096 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-25T16:57:17.178 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T16:57:17.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-25T16:57:17.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:57:17.253 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-04-25T16:57:17.330 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-25T16:57:17.330 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T16:57:17.341 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T16:57:17.341 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:57:17.341 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:57:17.344 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:57:17.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:57:17.346 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:57:17.346 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:57:17.374 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:57:17.375 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:57:17.389 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:57:17.389 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:57:17.399 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:57:17.412 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:57:17.413 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-04-25T16:57:17.414 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-04-25T16:57:17.424 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-04-25T16:57:17.425 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-04-25T16:57:17.435 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-04-25T16:57:17.437 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:57:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:57:17.446 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-04-25T16:57:17.451 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:57:17.451 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:57:17.460 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:57:17.473 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:57:17.503 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:57:17.504 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:57:17.507 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:57:17.507 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-25T16:57:17.507 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.507 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.507 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.508 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.508 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:57:17.509 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-25T16:57:17.509 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.509 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.509 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.509 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.526 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:57:17.530 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:57:17.530 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-25T16:57:17.530 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.530 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.530 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.530 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:57:17.531 INFO:teuthology.run_tasks:Running task install... 2024-04-25T16:57:17.540 DEBUG:teuthology.task.install:project ceph 2024-04-25T16:57:17.540 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T16:57:17.540 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T16:57:17.540 INFO:teuthology.task.install:Using flavor: default 2024-04-25T16:57:17.548 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T16:57:17.548 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T16:57:17.548 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:57:17.549 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:57:17.551 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:57:17.551 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:57:17.552 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:57:17.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:57:17.696 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:57:17.697 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:57:17.700 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:57:17.700 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:57:17.706 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:57:17.707 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:57:17.987 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:57:17.987 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T16:57:17.987 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:57:17.989 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:57:17.989 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T16:57:17.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:57:18.007 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:57:18.007 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T16:57:18.007 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:57:18.013 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:57:18.014 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:57:18.015 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:57:18.015 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:57:18.032 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:57:18.032 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:57:18.078 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:57:18.080 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:57:18.097 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:57:18.151 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:57:18.154 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:57:18.171 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:57:18.214 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-04-25T16:57:18.215 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-04-25T16:57:18.217 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-04-25T16:57:18.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-04-25T16:57:18.233 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-04-25T16:57:18.235 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-04-25T16:57:18.440 INFO:teuthology.orchestra.run.smithi171.stdout:48 files removed 2024-04-25T16:57:18.443 INFO:teuthology.orchestra.run.smithi097.stdout:48 files removed 2024-04-25T16:57:18.458 INFO:teuthology.orchestra.run.smithi096.stdout:48 files removed 2024-04-25T16:57:18.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:57:18.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:57:18.485 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:57:19.353 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 134 kB/s | 84 kB 00:00 2024-04-25T16:57:19.362 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 136 kB/s | 84 kB 00:00 2024-04-25T16:57:19.454 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 115 kB/s | 84 kB 00:00 2024-04-25T16:57:19.778 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-25T16:57:19.779 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T16:57:19.813 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T16:57:20.257 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-04-25T16:57:20.289 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-04-25T16:57:20.337 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-04-25T16:57:20.951 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T16:57:21.066 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T16:57:21.231 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-25T16:57:23.535 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T16:57:23.584 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T16:57:24.349 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 19 MB 00:02 2024-04-25T16:57:27.676 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T16:57:27.927 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-04-25T16:57:28.458 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T16:57:29.183 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-25T16:57:30.107 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-25T16:57:30.284 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-25T16:57:30.385 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-25T16:57:31.819 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T16:57:33.196 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 21 MB 00:02 2024-04-25T16:57:35.351 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T16:57:36.866 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T16:57:37.486 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:57:37.487 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:57:37.497 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:57:37.498 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:57:37.499 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:57:37.500 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:57:37.501 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:57:37.503 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:57:37.504 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:57:37.565 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:57:37.587 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:57:37.588 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:57:37.589 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:57:37.590 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:57:37.591 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:57:37.592 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:57:37.593 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:57:37.594 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:57:37.595 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:57:37.596 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:57:37.597 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:57:37.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:57:37.599 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:57:37.600 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:57:37.601 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout:Install 148 Packages 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 4 Packages 2024-04-25T16:57:37.602 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:57:37.603 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 214 M 2024-04-25T16:57:37.603 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-04-25T16:57:38.326 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-25T16:57:39.042 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:57:39.043 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:57:39.052 INFO:teuthology.orchestra.run.smithi171.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T16:57:39.054 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:57:39.055 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:57:39.056 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:57:39.056 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:57:39.058 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:57:39.060 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:57:39.061 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:57:39.121 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-25T16:57:39.138 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T16:57:39.138 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-04-25T16:57:39.138 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:57:39.139 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:57:39.140 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:57:39.141 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:57:39.142 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:57:39.143 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:57:39.144 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:57:39.145 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:57:39.146 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:57:39.147 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:57:39.148 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:57:39.149 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:57:39.150 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:57:39.151 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 4 Packages 2024-04-25T16:57:39.152 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:57:39.153 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 214 M 2024-04-25T16:57:39.153 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-04-25T16:57:39.611 INFO:teuthology.orchestra.run.smithi171.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.5 MB/s | 863 kB 00:00 2024-04-25T16:57:39.711 INFO:teuthology.orchestra.run.smithi171.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.4 MB/s | 147 kB 00:00 2024-04-25T16:57:40.136 INFO:teuthology.orchestra.run.smithi171.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-25T16:57:40.370 INFO:teuthology.orchestra.run.smithi171.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 3.4 MB/s | 2.3 MB 00:00 2024-04-25T16:57:40.404 INFO:teuthology.orchestra.run.smithi171.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-25T16:57:40.476 INFO:teuthology.orchestra.run.smithi097.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 10 kB/s | 6.6 kB 00:00 2024-04-25T16:57:40.485 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:57:40.486 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:57:40.496 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:57:40.497 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:57:40.498 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:57:40.498 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:57:40.500 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:57:40.502 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:57:40.503 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:57:40.564 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:57:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:57:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:57:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:57:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:57:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:57:40.578 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:57:40.579 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:57:40.580 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:57:40.581 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:57:40.582 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:57:40.583 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:57:40.584 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T16:57:40.584 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T16:57:40.584 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-04-25T16:57:40.584 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 4 Packages 2024-04-25T16:57:40.584 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:57:40.584 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 214 M 2024-04-25T16:57:40.584 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-25T16:57:41.387 INFO:teuthology.orchestra.run.smithi171.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 4.8 MB/s | 4.9 MB 00:01 2024-04-25T16:57:41.401 INFO:teuthology.orchestra.run.smithi097.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 933 kB/s | 863 kB 00:00 2024-04-25T16:57:41.603 INFO:teuthology.orchestra.run.smithi097.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 727 kB/s | 147 kB 00:00 2024-04-25T16:57:41.827 INFO:teuthology.orchestra.run.smithi096.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-25T16:57:42.396 INFO:teuthology.orchestra.run.smithi171.stdout:(8/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x86 8.9 MB/s | 18 MB 00:01 2024-04-25T16:57:42.403 INFO:teuthology.orchestra.run.smithi097.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-25T16:57:42.454 INFO:teuthology.orchestra.run.smithi171.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 434 kB/s | 25 kB 00:00 2024-04-25T16:57:42.896 INFO:teuthology.orchestra.run.smithi097.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T16:57:42.927 INFO:teuthology.orchestra.run.smithi096.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 784 kB/s | 863 kB 00:01 2024-04-25T16:57:42.946 INFO:teuthology.orchestra.run.smithi097.stdout:(6/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-04-25T16:57:42.954 INFO:teuthology.orchestra.run.smithi171.stdout:(10/152): ceph-common-19.0.0-3291.g994e2f20.el9 4.6 MB/s | 20 MB 00:04 2024-04-25T16:57:43.071 INFO:teuthology.orchestra.run.smithi171.stdout:(11/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 6.0 MB/s | 10 MB 00:01 2024-04-25T16:57:43.077 INFO:teuthology.orchestra.run.smithi096.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 981 kB/s | 147 kB 00:00 2024-04-25T16:57:43.096 INFO:teuthology.orchestra.run.smithi171.stdout:(12/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.5 MB/s | 224 kB 00:00 2024-04-25T16:57:43.138 INFO:teuthology.orchestra.run.smithi171.stdout:(13/152): cephfs-shell-19.0.0-3291.g994e2f20.el 405 kB/s | 27 kB 00:00 2024-04-25T16:57:43.164 INFO:teuthology.orchestra.run.smithi171.stdout:(14/152): libcephfs-devel-19.0.0-3291.g994e2f20 473 kB/s | 32 kB 00:00 2024-04-25T16:57:43.264 INFO:teuthology.orchestra.run.smithi171.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.5 MB/s | 158 kB 00:00 2024-04-25T16:57:43.323 INFO:teuthology.orchestra.run.smithi171.stdout:(16/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.0 MB/s | 756 kB 00:00 2024-04-25T16:57:43.356 INFO:teuthology.orchestra.run.smithi171.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.3 MB/s | 127 kB 00:00 2024-04-25T16:57:43.506 INFO:teuthology.orchestra.run.smithi171.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 2.6 MB/s | 497 kB 00:00 2024-04-25T16:57:43.607 INFO:teuthology.orchestra.run.smithi171.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 451 kB/s | 45 kB 00:00 2024-04-25T16:57:43.724 INFO:teuthology.orchestra.run.smithi171.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.1 MB/s | 135 kB 00:00 2024-04-25T16:57:43.850 INFO:teuthology.orchestra.run.smithi171.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.3 MB/s | 161 kB 00:00 2024-04-25T16:57:44.179 INFO:teuthology.orchestra.run.smithi097.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 3.8 MB/s | 4.9 MB 00:01 2024-04-25T16:57:44.192 INFO:teuthology.orchestra.run.smithi171.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 939 kB/s | 321 kB 00:00 2024-04-25T16:57:44.267 INFO:teuthology.orchestra.run.smithi171.stdout:(23/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 5.7 MB/s | 5.2 MB 00:00 2024-04-25T16:57:44.342 INFO:teuthology.orchestra.run.smithi171.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.9 MB/s | 298 kB 00:00 2024-04-25T16:57:44.367 INFO:teuthology.orchestra.run.smithi171.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 995 kB/s | 100 kB 00:00 2024-04-25T16:57:44.418 INFO:teuthology.orchestra.run.smithi171.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 86 kB 00:00 2024-04-25T16:57:44.427 INFO:teuthology.orchestra.run.smithi096.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-25T16:57:44.501 INFO:teuthology.orchestra.run.smithi171.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.0 MB/s | 169 kB 00:00 2024-04-25T16:57:44.593 INFO:teuthology.orchestra.run.smithi171.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 373 kB/s | 34 kB 00:00 2024-04-25T16:57:44.685 INFO:teuthology.orchestra.run.smithi171.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.5 MB/s | 142 kB 00:00 2024-04-25T16:57:44.927 INFO:teuthology.orchestra.run.smithi171.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 5.4 MB/s | 3.0 MB 00:00 2024-04-25T16:57:45.011 INFO:teuthology.orchestra.run.smithi096.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-04-25T16:57:45.119 INFO:teuthology.orchestra.run.smithi096.stdout:(6/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 1.1 MB/s | 2.3 MB 00:02 2024-04-25T16:57:45.636 INFO:teuthology.orchestra.run.smithi171.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 3.7 MB/s | 3.5 MB 00:00 2024-04-25T16:57:45.744 INFO:teuthology.orchestra.run.smithi171.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.2 MB/s | 249 kB 00:00 2024-04-25T16:57:45.844 INFO:teuthology.orchestra.run.smithi171.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 494 kB/s | 49 kB 00:00 2024-04-25T16:57:45.945 INFO:teuthology.orchestra.run.smithi171.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 164 kB/s | 16 kB 00:00 2024-04-25T16:57:46.053 INFO:teuthology.orchestra.run.smithi171.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.5 MB/s | 273 kB 00:00 2024-04-25T16:57:46.354 INFO:teuthology.orchestra.run.smithi171.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-25T16:57:46.387 INFO:teuthology.orchestra.run.smithi171.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.2 MB/s | 762 kB 00:00 2024-04-25T16:57:46.404 INFO:teuthology.orchestra.run.smithi171.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 365 kB/s | 18 kB 00:00 2024-04-25T16:57:46.521 INFO:teuthology.orchestra.run.smithi171.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 602 kB/s | 80 kB 00:00 2024-04-25T16:57:46.580 INFO:teuthology.orchestra.run.smithi171.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.3 MB/s | 803 kB 00:00 2024-04-25T16:57:46.605 INFO:teuthology.orchestra.run.smithi171.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T16:57:46.628 INFO:teuthology.orchestra.run.smithi096.stdout:(7/152): ceph-common-19.0.0-3291.g994e2f20.el9. 3.8 MB/s | 20 MB 00:05 2024-04-25T16:57:46.630 INFO:teuthology.orchestra.run.smithi171.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T16:57:46.711 INFO:teuthology.orchestra.run.smithi096.stdout:(8/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 2.9 MB/s | 4.9 MB 00:01 2024-04-25T16:57:46.714 INFO:teuthology.orchestra.run.smithi171.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T16:57:46.739 INFO:teuthology.orchestra.run.smithi171.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 987 kB/s | 106 kB 00:00 2024-04-25T16:57:46.773 INFO:teuthology.orchestra.run.smithi171.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T16:57:46.778 INFO:teuthology.orchestra.run.smithi096.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 379 kB/s | 25 kB 00:00 2024-04-25T16:57:46.798 INFO:teuthology.orchestra.run.smithi171.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T16:57:46.823 INFO:teuthology.orchestra.run.smithi171.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T16:57:46.882 INFO:teuthology.orchestra.run.smithi171.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-04-25T16:57:46.924 INFO:teuthology.orchestra.run.smithi171.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-04-25T16:57:46.950 INFO:teuthology.orchestra.run.smithi171.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T16:57:46.978 INFO:teuthology.orchestra.run.smithi171.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T16:57:47.000 INFO:teuthology.orchestra.run.smithi171.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T16:57:47.025 INFO:teuthology.orchestra.run.smithi171.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 659 kB/s | 33 kB 00:00 2024-04-25T16:57:47.076 INFO:teuthology.orchestra.run.smithi171.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 370 kB/s | 18 kB 00:00 2024-04-25T16:57:47.201 INFO:teuthology.orchestra.run.smithi171.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-25T16:57:47.218 INFO:teuthology.orchestra.run.smithi171.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 79 kB/s | 11 kB 00:00 2024-04-25T16:57:47.252 INFO:teuthology.orchestra.run.smithi171.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T16:57:47.277 INFO:teuthology.orchestra.run.smithi171.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T16:57:47.310 INFO:teuthology.orchestra.run.smithi171.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T16:57:47.336 INFO:teuthology.orchestra.run.smithi171.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T16:57:47.353 INFO:teuthology.orchestra.run.smithi171.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 625 kB/s | 25 kB 00:00 2024-04-25T16:57:47.370 INFO:teuthology.orchestra.run.smithi171.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T16:57:47.371 INFO:teuthology.orchestra.run.smithi097.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 2.6 MB/s | 20 MB 00:07 2024-04-25T16:57:47.395 INFO:teuthology.orchestra.run.smithi171.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T16:57:47.420 INFO:teuthology.orchestra.run.smithi171.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T16:57:47.445 INFO:teuthology.orchestra.run.smithi171.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T16:57:47.462 INFO:teuthology.orchestra.run.smithi097.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 275 kB/s | 25 kB 00:00 2024-04-25T16:57:47.713 INFO:teuthology.orchestra.run.smithi171.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-04-25T16:57:47.796 INFO:teuthology.orchestra.run.smithi171.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T16:57:47.830 INFO:teuthology.orchestra.run.smithi171.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T16:57:47.872 INFO:teuthology.orchestra.run.smithi171.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T16:57:47.897 INFO:teuthology.orchestra.run.smithi171.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T16:57:48.022 INFO:teuthology.orchestra.run.smithi171.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-25T16:57:48.029 INFO:teuthology.orchestra.run.smithi097.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 2.6 MB/s | 10 MB 00:03 2024-04-25T16:57:48.146 INFO:teuthology.orchestra.run.smithi097.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.9 MB/s | 224 kB 00:00 2024-04-25T16:57:48.147 INFO:teuthology.orchestra.run.smithi171.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-04-25T16:57:48.172 INFO:teuthology.orchestra.run.smithi171.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 592 kB/s | 87 kB 00:00 2024-04-25T16:57:48.213 INFO:teuthology.orchestra.run.smithi097.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 405 kB/s | 27 kB 00:00 2024-04-25T16:57:48.223 INFO:teuthology.orchestra.run.smithi171.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-25T16:57:48.286 INFO:teuthology.orchestra.run.smithi096.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 6.1 MB/s | 10 MB 00:01 2024-04-25T16:57:48.296 INFO:teuthology.orchestra.run.smithi097.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T16:57:48.323 INFO:teuthology.orchestra.run.smithi171.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.5 MB/s | 248 kB 00:00 2024-04-25T16:57:48.407 INFO:teuthology.orchestra.run.smithi171.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-04-25T16:57:48.411 INFO:teuthology.orchestra.run.smithi096.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.8 MB/s | 224 kB 00:00 2024-04-25T16:57:48.441 INFO:teuthology.orchestra.run.smithi171.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 220 kB/s | 26 kB 00:00 2024-04-25T16:57:48.466 INFO:teuthology.orchestra.run.smithi171.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 325 kB/s | 19 kB 00:00 2024-04-25T16:57:48.470 INFO:teuthology.orchestra.run.smithi096.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 465 kB/s | 27 kB 00:00 2024-04-25T16:57:48.499 INFO:teuthology.orchestra.run.smithi171.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T16:57:48.525 INFO:teuthology.orchestra.run.smithi171.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-25T16:57:48.530 INFO:teuthology.orchestra.run.smithi097.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.2 MB/s | 756 kB 00:00 2024-04-25T16:57:48.537 INFO:teuthology.orchestra.run.smithi096.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 475 kB/s | 32 kB 00:00 2024-04-25T16:57:48.600 INFO:teuthology.orchestra.run.smithi171.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-25T16:57:48.647 INFO:teuthology.orchestra.run.smithi097.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.3 MB/s | 158 kB 00:00 2024-04-25T16:57:48.696 INFO:teuthology.orchestra.run.smithi096.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.7 MB/s | 756 kB 00:00 2024-04-25T16:57:48.726 INFO:teuthology.orchestra.run.smithi171.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-25T16:57:48.731 INFO:teuthology.orchestra.run.smithi097.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.5 MB/s | 127 kB 00:00 2024-04-25T16:57:48.751 INFO:teuthology.orchestra.run.smithi171.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-25T16:57:48.754 INFO:teuthology.orchestra.run.smithi096.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.6 MB/s | 158 kB 00:00 2024-04-25T16:57:48.776 INFO:teuthology.orchestra.run.smithi171.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T16:57:48.801 INFO:teuthology.orchestra.run.smithi171.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-04-25T16:57:48.813 INFO:teuthology.orchestra.run.smithi096.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 2.1 MB/s | 127 kB 00:00 2024-04-25T16:57:48.827 INFO:teuthology.orchestra.run.smithi171.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T16:57:48.852 INFO:teuthology.orchestra.run.smithi171.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T16:57:48.873 INFO:teuthology.orchestra.run.smithi097.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.4 MB/s | 497 kB 00:00 2024-04-25T16:57:48.877 INFO:teuthology.orchestra.run.smithi171.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T16:57:48.894 INFO:teuthology.orchestra.run.smithi171.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T16:57:48.921 INFO:teuthology.orchestra.run.smithi096.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 4.5 MB/s | 497 kB 00:00 2024-04-25T16:57:48.928 INFO:teuthology.orchestra.run.smithi171.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T16:57:48.961 INFO:teuthology.orchestra.run.smithi171.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.3 MB/s | 42 kB 00:00 2024-04-25T16:57:49.003 INFO:teuthology.orchestra.run.smithi171.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-25T16:57:49.062 INFO:teuthology.orchestra.run.smithi171.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T16:57:49.137 INFO:teuthology.orchestra.run.smithi171.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 494 kB/s | 36 kB 00:00 2024-04-25T16:57:49.354 INFO:teuthology.orchestra.run.smithi171.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T16:57:49.371 INFO:teuthology.orchestra.run.smithi171.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 205 kB/s | 47 kB 00:00 2024-04-25T16:57:49.413 INFO:teuthology.orchestra.run.smithi171.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T16:57:49.430 INFO:teuthology.orchestra.run.smithi171.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 982 kB/s | 58 kB 00:00 2024-04-25T16:57:49.455 INFO:teuthology.orchestra.run.smithi171.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-25T16:57:49.473 INFO:teuthology.orchestra.run.smithi171.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-25T16:57:49.489 INFO:teuthology.orchestra.run.smithi171.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T16:57:49.506 INFO:teuthology.orchestra.run.smithi171.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 668 kB/s | 22 kB 00:00 2024-04-25T16:57:49.680 INFO:teuthology.orchestra.run.smithi096.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 6.8 MB/s | 5.2 MB 00:00 2024-04-25T16:57:49.730 INFO:teuthology.orchestra.run.smithi096.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 909 kB/s | 45 kB 00:00 2024-04-25T16:57:49.789 INFO:teuthology.orchestra.run.smithi096.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.3 MB/s | 135 kB 00:00 2024-04-25T16:57:49.847 INFO:teuthology.orchestra.run.smithi096.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.7 MB/s | 161 kB 00:00 2024-04-25T16:57:49.906 INFO:teuthology.orchestra.run.smithi096.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 5.4 MB/s | 321 kB 00:00 2024-04-25T16:57:49.948 INFO:teuthology.orchestra.run.smithi171.stdout:(103/152): ceph-test-19.0.0-3291.g994e2f20.el9. 5.8 MB/s | 43 MB 00:07 2024-04-25T16:57:49.990 INFO:teuthology.orchestra.run.smithi096.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.5 MB/s | 298 kB 00:00 2024-04-25T16:57:50.007 INFO:teuthology.orchestra.run.smithi171.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-04-25T16:57:50.040 INFO:teuthology.orchestra.run.smithi096.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 2.0 MB/s | 100 kB 00:00 2024-04-25T16:57:50.090 INFO:teuthology.orchestra.run.smithi097.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 4.2 MB/s | 5.2 MB 00:01 2024-04-25T16:57:50.090 INFO:teuthology.orchestra.run.smithi096.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.7 MB/s | 86 kB 00:00 2024-04-25T16:57:50.148 INFO:teuthology.orchestra.run.smithi097.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 778 kB/s | 45 kB 00:00 2024-04-25T16:57:50.232 INFO:teuthology.orchestra.run.smithi097.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.6 MB/s | 135 kB 00:00 2024-04-25T16:57:50.432 INFO:teuthology.orchestra.run.smithi097.stdout:(21/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 2.4 MB/s | 18 MB 00:07 2024-04-25T16:57:50.449 INFO:teuthology.orchestra.run.smithi096.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 8.5 MB/s | 3.0 MB 00:00 2024-04-25T16:57:50.457 INFO:teuthology.orchestra.run.smithi097.stdout:(22/152): python3-cephfs-19.0.0-3291.g994e2f20. 717 kB/s | 161 kB 00:00 2024-04-25T16:57:50.499 INFO:teuthology.orchestra.run.smithi096.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 169 kB 00:00 2024-04-25T16:57:50.549 INFO:teuthology.orchestra.run.smithi097.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 2.7 MB/s | 321 kB 00:00 2024-04-25T16:57:50.558 INFO:teuthology.orchestra.run.smithi096.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 585 kB/s | 34 kB 00:00 2024-04-25T16:57:50.574 INFO:teuthology.orchestra.run.smithi171.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 34 kB/s | 19 kB 00:00 2024-04-25T16:57:50.575 INFO:teuthology.orchestra.run.smithi097.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.5 MB/s | 298 kB 00:00 2024-04-25T16:57:50.608 INFO:teuthology.orchestra.run.smithi097.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 100 kB 00:00 2024-04-25T16:57:50.617 INFO:teuthology.orchestra.run.smithi096.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.4 MB/s | 142 kB 00:00 2024-04-25T16:57:50.642 INFO:teuthology.orchestra.run.smithi097.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 86 kB 00:00 2024-04-25T16:57:50.707 INFO:teuthology.orchestra.run.smithi171.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 422 kB/s | 309 kB 00:00 2024-04-25T16:57:50.717 INFO:teuthology.orchestra.run.smithi097.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.2 MB/s | 169 kB 00:00 2024-04-25T16:57:50.783 INFO:teuthology.orchestra.run.smithi171.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-04-25T16:57:50.801 INFO:teuthology.orchestra.run.smithi097.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 411 kB/s | 34 kB 00:00 2024-04-25T16:57:50.808 INFO:teuthology.orchestra.run.smithi171.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 421 kB/s | 548 kB 00:01 2024-04-25T16:57:50.907 INFO:teuthology.orchestra.run.smithi097.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.3 MB/s | 142 kB 00:00 2024-04-25T16:57:51.050 INFO:teuthology.orchestra.run.smithi096.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 8.2 MB/s | 3.5 MB 00:00 2024-04-25T16:57:51.208 INFO:teuthology.orchestra.run.smithi171.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 168 kB/s | 67 kB 00:00 2024-04-25T16:57:51.233 INFO:teuthology.orchestra.run.smithi171.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 108 kB/s | 49 kB 00:00 2024-04-25T16:57:51.425 INFO:teuthology.orchestra.run.smithi171.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-04-25T16:57:51.501 INFO:teuthology.orchestra.run.smithi171.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-25T16:57:51.560 INFO:teuthology.orchestra.run.smithi097.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.2 MB/s | 3.0 MB 00:00 2024-04-25T16:57:51.643 INFO:teuthology.orchestra.run.smithi171.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-25T16:57:51.676 INFO:teuthology.orchestra.run.smithi171.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-04-25T16:57:51.692 INFO:teuthology.orchestra.run.smithi096.stdout:(31/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 2.7 MB/s | 18 MB 00:06 2024-04-25T16:57:51.694 INFO:teuthology.orchestra.run.smithi097.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 4.5 MB/s | 3.5 MB 00:00 2024-04-25T16:57:51.710 INFO:teuthology.orchestra.run.smithi171.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-25T16:57:51.743 INFO:teuthology.orchestra.run.smithi171.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-25T16:57:51.786 INFO:teuthology.orchestra.run.smithi097.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.7 MB/s | 249 kB 00:00 2024-04-25T16:57:51.801 INFO:teuthology.orchestra.run.smithi096.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.3 MB/s | 249 kB 00:00 2024-04-25T16:57:51.810 INFO:teuthology.orchestra.run.smithi171.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T16:57:51.844 INFO:teuthology.orchestra.run.smithi171.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-25T16:57:51.859 INFO:teuthology.orchestra.run.smithi096.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 846 kB/s | 49 kB 00:00 2024-04-25T16:57:51.861 INFO:teuthology.orchestra.run.smithi097.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 657 kB/s | 49 kB 00:00 2024-04-25T16:57:51.909 INFO:teuthology.orchestra.run.smithi096.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 330 kB/s | 16 kB 00:00 2024-04-25T16:57:51.936 INFO:teuthology.orchestra.run.smithi097.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 220 kB/s | 16 kB 00:00 2024-04-25T16:57:51.978 INFO:teuthology.orchestra.run.smithi171.stdout:(119/152): luarocks-3.9.2-1.el9.noarch.rpm 196 kB/s | 151 kB 00:00 2024-04-25T16:57:52.011 INFO:teuthology.orchestra.run.smithi097.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 3.6 MB/s | 273 kB 00:00 2024-04-25T16:57:52.026 INFO:teuthology.orchestra.run.smithi096.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-04-25T16:57:52.044 INFO:teuthology.orchestra.run.smithi171.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.3 MB/s | 303 kB 00:00 2024-04-25T16:57:52.052 INFO:teuthology.orchestra.run.smithi096.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.9 MB/s | 273 kB 00:00 2024-04-25T16:57:52.070 INFO:teuthology.orchestra.run.smithi171.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 156 kB/s | 35 kB 00:00 2024-04-25T16:57:52.111 INFO:teuthology.orchestra.run.smithi096.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 313 kB/s | 18 kB 00:00 2024-04-25T16:57:52.144 INFO:teuthology.orchestra.run.smithi096.stdout:(38/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-25T16:57:52.187 INFO:teuthology.orchestra.run.smithi097.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 4.3 MB/s | 762 kB 00:00 2024-04-25T16:57:52.254 INFO:teuthology.orchestra.run.smithi097.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 273 kB/s | 18 kB 00:00 2024-04-25T16:57:52.269 INFO:teuthology.orchestra.run.smithi096.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 507 kB/s | 80 kB 00:00 2024-04-25T16:57:52.328 INFO:teuthology.orchestra.run.smithi096.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-04-25T16:57:52.353 INFO:teuthology.orchestra.run.smithi096.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T16:57:52.362 INFO:teuthology.orchestra.run.smithi097.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T16:57:52.378 INFO:teuthology.orchestra.run.smithi096.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T16:57:52.454 INFO:teuthology.orchestra.run.smithi096.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-25T16:57:52.471 INFO:teuthology.orchestra.run.smithi096.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T16:57:52.478 INFO:teuthology.orchestra.run.smithi171.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.7 MB/s | 2.0 MB 00:00 2024-04-25T16:57:52.479 INFO:teuthology.orchestra.run.smithi097.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-25T16:57:52.496 INFO:teuthology.orchestra.run.smithi096.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T16:57:52.504 INFO:teuthology.orchestra.run.smithi171.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-04-25T16:57:52.521 INFO:teuthology.orchestra.run.smithi097.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T16:57:52.521 INFO:teuthology.orchestra.run.smithi096.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T16:57:52.529 INFO:teuthology.orchestra.run.smithi171.stdout:(124/152): python3-google-auth-2.29.0-1.el9.noa 373 kB/s | 206 kB 00:00 2024-04-25T16:57:52.546 INFO:teuthology.orchestra.run.smithi096.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T16:57:52.554 INFO:teuthology.orchestra.run.smithi171.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-25T16:57:52.563 INFO:teuthology.orchestra.run.smithi097.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T16:57:52.572 INFO:teuthology.orchestra.run.smithi096.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T16:57:52.579 INFO:teuthology.orchestra.run.smithi171.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-25T16:57:52.605 INFO:teuthology.orchestra.run.smithi171.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-25T16:57:52.613 INFO:teuthology.orchestra.run.smithi096.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T16:57:52.638 INFO:teuthology.orchestra.run.smithi171.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-25T16:57:52.639 INFO:teuthology.orchestra.run.smithi096.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T16:57:52.655 INFO:teuthology.orchestra.run.smithi097.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-25T16:57:52.663 INFO:teuthology.orchestra.run.smithi171.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-25T16:57:52.664 INFO:teuthology.orchestra.run.smithi096.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T16:57:52.689 INFO:teuthology.orchestra.run.smithi096.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T16:57:52.697 INFO:teuthology.orchestra.run.smithi097.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T16:57:52.706 INFO:teuthology.orchestra.run.smithi096.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 791 kB/s | 33 kB 00:00 2024-04-25T16:57:52.730 INFO:teuthology.orchestra.run.smithi097.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T16:57:52.731 INFO:teuthology.orchestra.run.smithi096.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 744 kB/s | 18 kB 00:00 2024-04-25T16:57:52.781 INFO:teuthology.orchestra.run.smithi097.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T16:57:52.798 INFO:teuthology.orchestra.run.smithi096.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T16:57:52.805 INFO:teuthology.orchestra.run.smithi171.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 277 kB/s | 46 kB 00:00 2024-04-25T16:57:52.815 INFO:teuthology.orchestra.run.smithi096.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 136 kB/s | 11 kB 00:00 2024-04-25T16:57:52.823 INFO:teuthology.orchestra.run.smithi097.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T16:57:52.831 INFO:teuthology.orchestra.run.smithi171.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 472 kB/s | 79 kB 00:00 2024-04-25T16:57:52.841 INFO:teuthology.orchestra.run.smithi096.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T16:57:52.864 INFO:teuthology.orchestra.run.smithi171.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-04-25T16:57:52.866 INFO:teuthology.orchestra.run.smithi096.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-25T16:57:52.873 INFO:teuthology.orchestra.run.smithi097.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T16:57:52.891 INFO:teuthology.orchestra.run.smithi096.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T16:57:52.914 INFO:teuthology.orchestra.run.smithi171.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-25T16:57:52.925 INFO:teuthology.orchestra.run.smithi096.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 10 MB/s | 582 kB 00:00 2024-04-25T16:57:52.931 INFO:teuthology.orchestra.run.smithi171.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-25T16:57:52.940 INFO:teuthology.orchestra.run.smithi097.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T16:57:52.951 INFO:teuthology.orchestra.run.smithi096.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 442 kB/s | 25 kB 00:00 2024-04-25T16:57:52.965 INFO:teuthology.orchestra.run.smithi171.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-25T16:57:52.974 INFO:teuthology.orchestra.run.smithi097.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T16:57:52.976 INFO:teuthology.orchestra.run.smithi096.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T16:57:52.999 INFO:teuthology.orchestra.run.smithi171.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-25T16:57:53.007 INFO:teuthology.orchestra.run.smithi097.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 650 kB/s | 21 kB 00:00 2024-04-25T16:57:53.009 INFO:teuthology.orchestra.run.smithi096.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T16:57:53.025 INFO:teuthology.orchestra.run.smithi171.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-25T16:57:53.035 INFO:teuthology.orchestra.run.smithi096.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T16:57:53.049 INFO:teuthology.orchestra.run.smithi171.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 366 kB/s | 31 kB 00:00 2024-04-25T16:57:53.050 INFO:teuthology.orchestra.run.smithi097.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T16:57:53.060 INFO:teuthology.orchestra.run.smithi096.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 709 kB/s | 35 kB 00:00 2024-04-25T16:57:53.100 INFO:teuthology.orchestra.run.smithi097.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-04-25T16:57:53.168 INFO:teuthology.orchestra.run.smithi096.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-04-25T16:57:53.208 INFO:teuthology.orchestra.run.smithi171.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 902 kB/s | 188 kB 00:00 2024-04-25T16:57:53.250 INFO:teuthology.orchestra.run.smithi097.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-04-25T16:57:53.260 INFO:teuthology.orchestra.run.smithi096.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-25T16:57:53.275 INFO:teuthology.orchestra.run.smithi171.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 159 kB/s | 36 kB 00:00 2024-04-25T16:57:53.300 INFO:teuthology.orchestra.run.smithi171.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-25T16:57:53.302 INFO:teuthology.orchestra.run.smithi096.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T16:57:53.334 INFO:teuthology.orchestra.run.smithi097.stdout:(54/152): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-04-25T16:57:53.353 INFO:teuthology.orchestra.run.smithi096.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 10 MB/s | 911 kB 00:00 2024-04-25T16:57:53.359 INFO:teuthology.orchestra.run.smithi097.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-25T16:57:53.378 INFO:teuthology.orchestra.run.smithi096.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-25T16:57:53.384 INFO:teuthology.orchestra.run.smithi097.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T16:57:53.409 INFO:teuthology.orchestra.run.smithi097.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T16:57:53.434 INFO:teuthology.orchestra.run.smithi097.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T16:57:53.453 INFO:teuthology.orchestra.run.smithi096.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-04-25T16:57:53.467 INFO:teuthology.orchestra.run.smithi171.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-25T16:57:53.477 INFO:teuthology.orchestra.run.smithi097.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T16:57:53.492 INFO:teuthology.orchestra.run.smithi171.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 467 kB/s | 90 kB 00:00 2024-04-25T16:57:53.502 INFO:teuthology.orchestra.run.smithi097.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 582 kB 00:00 2024-04-25T16:57:53.536 INFO:teuthology.orchestra.run.smithi097.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 440 kB/s | 25 kB 00:00 2024-04-25T16:57:53.553 INFO:teuthology.orchestra.run.smithi096.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T16:57:53.561 INFO:teuthology.orchestra.run.smithi097.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-04-25T16:57:53.579 INFO:teuthology.orchestra.run.smithi096.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 711 kB/s | 87 kB 00:00 2024-04-25T16:57:53.586 INFO:teuthology.orchestra.run.smithi097.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T16:57:53.611 INFO:teuthology.orchestra.run.smithi097.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T16:57:53.620 INFO:teuthology.orchestra.run.smithi096.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-25T16:57:53.653 INFO:teuthology.orchestra.run.smithi097.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 532 kB/s | 35 kB 00:00 2024-04-25T16:57:53.704 INFO:teuthology.orchestra.run.smithi096.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.0 MB/s | 248 kB 00:00 2024-04-25T16:57:53.718 INFO:teuthology.orchestra.run.smithi171.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 89 kB/s | 20 kB 00:00 2024-04-25T16:57:53.743 INFO:teuthology.orchestra.run.smithi171.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.5 MB/s | 427 kB 00:00 2024-04-25T16:57:53.780 INFO:teuthology.orchestra.run.smithi096.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-25T16:57:53.805 INFO:teuthology.orchestra.run.smithi096.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 261 kB/s | 26 kB 00:00 2024-04-25T16:57:53.812 INFO:teuthology.orchestra.run.smithi097.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-04-25T16:57:53.830 INFO:teuthology.orchestra.run.smithi096.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T16:57:53.879 INFO:teuthology.orchestra.run.smithi097.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-25T16:57:53.912 INFO:teuthology.orchestra.run.smithi096.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-04-25T16:57:53.912 INFO:teuthology.orchestra.run.smithi097.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T16:57:53.939 INFO:teuthology.orchestra.run.smithi096.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-25T16:57:53.981 INFO:teuthology.orchestra.run.smithi096.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.0 MB/s | 250 kB 00:00 2024-04-25T16:57:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.4 MB/s | 911 kB 00:00 2024-04-25T16:57:54.030 INFO:teuthology.orchestra.run.smithi097.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T16:57:54.065 INFO:teuthology.orchestra.run.smithi096.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 33 MB/s | 5.1 MB 00:00 2024-04-25T16:57:54.090 INFO:teuthology.orchestra.run.smithi096.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-25T16:57:54.105 INFO:teuthology.orchestra.run.smithi097.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-04-25T16:57:54.115 INFO:teuthology.orchestra.run.smithi096.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T16:57:54.141 INFO:teuthology.orchestra.run.smithi096.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T16:57:54.166 INFO:teuthology.orchestra.run.smithi096.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T16:57:54.189 INFO:teuthology.orchestra.run.smithi097.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-04-25T16:57:54.191 INFO:teuthology.orchestra.run.smithi096.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T16:57:54.214 INFO:teuthology.orchestra.run.smithi097.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 824 kB/s | 87 kB 00:00 2024-04-25T16:57:54.216 INFO:teuthology.orchestra.run.smithi096.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T16:57:54.233 INFO:teuthology.orchestra.run.smithi096.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T16:57:54.267 INFO:teuthology.orchestra.run.smithi096.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T16:57:54.334 INFO:teuthology.orchestra.run.smithi096.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-04-25T16:57:54.376 INFO:teuthology.orchestra.run.smithi171.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 291 kB/s | 192 kB 00:00 2024-04-25T16:57:54.381 INFO:teuthology.orchestra.run.smithi097.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-04-25T16:57:54.393 INFO:teuthology.orchestra.run.smithi096.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 784 kB/s | 44 kB 00:00 2024-04-25T16:57:54.431 INFO:teuthology.orchestra.run.smithi097.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T16:57:54.457 INFO:teuthology.orchestra.run.smithi097.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T16:57:54.468 INFO:teuthology.orchestra.run.smithi096.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-25T16:57:54.482 INFO:teuthology.orchestra.run.smithi097.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-04-25T16:57:54.507 INFO:teuthology.orchestra.run.smithi097.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T16:57:54.540 INFO:teuthology.orchestra.run.smithi097.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T16:57:54.557 INFO:teuthology.orchestra.run.smithi097.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-25T16:57:54.608 INFO:teuthology.orchestra.run.smithi097.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T16:57:54.668 INFO:teuthology.orchestra.run.smithi096.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-25T16:57:54.685 INFO:teuthology.orchestra.run.smithi096.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 167 kB/s | 36 kB 00:00 2024-04-25T16:57:54.702 INFO:teuthology.orchestra.run.smithi096.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T16:57:54.717 INFO:teuthology.orchestra.run.smithi097.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T16:57:54.727 INFO:teuthology.orchestra.run.smithi096.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-04-25T16:57:54.742 INFO:teuthology.orchestra.run.smithi097.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-04-25T16:57:54.752 INFO:teuthology.orchestra.run.smithi096.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T16:57:54.767 INFO:teuthology.orchestra.run.smithi097.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T16:57:54.778 INFO:teuthology.orchestra.run.smithi096.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T16:57:54.793 INFO:teuthology.orchestra.run.smithi097.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-25T16:57:54.802 INFO:teuthology.orchestra.run.smithi171.stdout:(147/152): python3-rsa-4.9-2.el9.noarch.rpm 33 kB/s | 59 kB 00:01 2024-04-25T16:57:54.803 INFO:teuthology.orchestra.run.smithi096.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T16:57:54.817 INFO:teuthology.orchestra.run.smithi097.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T16:57:54.828 INFO:teuthology.orchestra.run.smithi096.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-04-25T16:57:54.843 INFO:teuthology.orchestra.run.smithi097.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T16:57:54.853 INFO:teuthology.orchestra.run.smithi096.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-25T16:57:54.868 INFO:teuthology.orchestra.run.smithi097.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T16:57:54.893 INFO:teuthology.orchestra.run.smithi097.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T16:57:54.896 INFO:teuthology.orchestra.run.smithi096.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T16:57:54.919 INFO:teuthology.orchestra.run.smithi097.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T16:57:54.935 INFO:teuthology.orchestra.run.smithi171.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-04-25T16:57:55.019 INFO:teuthology.orchestra.run.smithi171.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T16:57:55.136 INFO:teuthology.orchestra.run.smithi171.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T16:57:55.344 INFO:teuthology.orchestra.run.smithi097.stdout:(91/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 5.5 MB/s | 43 MB 00:07 2024-04-25T16:57:55.354 INFO:teuthology.orchestra.run.smithi096.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 674 kB/s | 309 kB 00:00 2024-04-25T16:57:55.369 INFO:teuthology.orchestra.run.smithi097.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 93 kB/s | 42 kB 00:00 2024-04-25T16:57:55.404 INFO:teuthology.orchestra.run.smithi096.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 997 kB/s | 548 kB 00:00 2024-04-25T16:57:55.455 INFO:teuthology.orchestra.run.smithi096.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-25T16:57:55.596 INFO:teuthology.orchestra.run.smithi096.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-04-25T16:57:55.628 INFO:teuthology.orchestra.run.smithi171.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-25T16:57:55.645 INFO:teuthology.orchestra.run.smithi097.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 148 kB/s | 44 kB 00:00 2024-04-25T16:57:55.670 INFO:teuthology.orchestra.run.smithi097.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 742 kB/s | 221 kB 00:00 2024-04-25T16:57:55.711 INFO:teuthology.orchestra.run.smithi171.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 2.5 MB/s | 3.3 MB 00:01 2024-04-25T16:57:55.713 INFO:teuthology.orchestra.run.smithi096.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-25T16:57:55.720 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:57:55.720 INFO:teuthology.orchestra.run.smithi171.stdout:Total 12 MB/s | 214 MB 00:18 2024-04-25T16:57:55.813 INFO:teuthology.orchestra.run.smithi096.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-04-25T16:57:55.829 INFO:teuthology.orchestra.run.smithi097.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-25T16:57:55.854 INFO:teuthology.orchestra.run.smithi097.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 172 kB/s | 36 kB 00:00 2024-04-25T16:57:55.879 INFO:teuthology.orchestra.run.smithi097.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 229 kB/s | 47 kB 00:00 2024-04-25T16:57:55.904 INFO:teuthology.orchestra.run.smithi097.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T16:57:55.914 INFO:teuthology.orchestra.run.smithi096.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-25T16:57:55.929 INFO:teuthology.orchestra.run.smithi097.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 771 kB/s | 58 kB 00:00 2024-04-25T16:57:55.955 INFO:teuthology.orchestra.run.smithi097.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T16:57:55.980 INFO:teuthology.orchestra.run.smithi097.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T16:57:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-04-25T16:57:56.030 INFO:teuthology.orchestra.run.smithi097.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-25T16:57:56.064 INFO:teuthology.orchestra.run.smithi097.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T16:57:56.089 INFO:teuthology.orchestra.run.smithi096.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-25T16:57:56.148 INFO:teuthology.orchestra.run.smithi097.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 234 kB/s | 19 kB 00:00 2024-04-25T16:57:56.198 INFO:teuthology.orchestra.run.smithi096.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-25T16:57:56.214 INFO:teuthology.orchestra.run.smithi097.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-25T16:57:56.248 INFO:teuthology.orchestra.run.smithi097.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-04-25T16:57:56.273 INFO:teuthology.orchestra.run.smithi097.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-04-25T16:57:56.290 INFO:teuthology.orchestra.run.smithi096.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-04-25T16:57:56.299 INFO:teuthology.orchestra.run.smithi097.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-04-25T16:57:56.315 INFO:teuthology.orchestra.run.smithi096.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-04-25T16:57:56.324 INFO:teuthology.orchestra.run.smithi097.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T16:57:56.349 INFO:teuthology.orchestra.run.smithi097.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-25T16:57:56.382 INFO:teuthology.orchestra.run.smithi096.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-04-25T16:57:56.407 INFO:teuthology.orchestra.run.smithi096.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T16:57:56.425 INFO:teuthology.orchestra.run.smithi097.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T16:57:56.450 INFO:teuthology.orchestra.run.smithi097.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.5 MB/s | 838 kB 00:00 2024-04-25T16:57:56.475 INFO:teuthology.orchestra.run.smithi097.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-04-25T16:57:56.482 INFO:teuthology.orchestra.run.smithi096.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T16:57:56.500 INFO:teuthology.orchestra.run.smithi097.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-25T16:57:56.508 INFO:teuthology.orchestra.run.smithi096.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-25T16:57:56.525 INFO:teuthology.orchestra.run.smithi097.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-25T16:57:56.551 INFO:teuthology.orchestra.run.smithi097.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T16:57:56.583 INFO:teuthology.orchestra.run.smithi096.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T16:57:56.608 INFO:teuthology.orchestra.run.smithi096.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-25T16:57:56.634 INFO:teuthology.orchestra.run.smithi097.stdout:(118/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-25T16:57:56.660 INFO:teuthology.orchestra.run.smithi097.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-25T16:57:56.677 INFO:teuthology.orchestra.run.smithi097.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 844 kB/s | 35 kB 00:00 2024-04-25T16:57:56.684 INFO:teuthology.orchestra.run.smithi096.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T16:57:56.702 INFO:teuthology.orchestra.run.smithi097.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-25T16:57:56.736 INFO:teuthology.orchestra.run.smithi097.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T16:57:56.761 INFO:teuthology.orchestra.run.smithi097.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T16:57:56.784 INFO:teuthology.orchestra.run.smithi096.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T16:57:56.786 INFO:teuthology.orchestra.run.smithi097.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-25T16:57:56.811 INFO:teuthology.orchestra.run.smithi097.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-25T16:57:56.827 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-04-25T16:57:56.837 INFO:teuthology.orchestra.run.smithi097.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-04-25T16:57:56.851 INFO:teuthology.orchestra.run.smithi096.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-04-25T16:57:56.862 INFO:teuthology.orchestra.run.smithi097.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-04-25T16:57:56.884 INFO:teuthology.orchestra.run.smithi096.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-25T16:57:56.887 INFO:teuthology.orchestra.run.smithi097.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-25T16:57:56.954 INFO:teuthology.orchestra.run.smithi097.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-04-25T16:57:56.968 INFO:teuthology.orchestra.run.smithi096.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T16:57:56.971 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-04-25T16:57:56.971 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-04-25T16:57:56.979 INFO:teuthology.orchestra.run.smithi097.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-04-25T16:57:56.993 INFO:teuthology.orchestra.run.smithi096.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-25T16:57:57.022 INFO:teuthology.orchestra.run.smithi097.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T16:57:57.047 INFO:teuthology.orchestra.run.smithi097.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-04-25T16:57:57.060 INFO:teuthology.orchestra.run.smithi096.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T16:57:57.072 INFO:teuthology.orchestra.run.smithi097.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.4 MB/s | 272 kB 00:00 2024-04-25T16:57:57.085 INFO:teuthology.orchestra.run.smithi096.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T16:57:57.097 INFO:teuthology.orchestra.run.smithi097.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-25T16:57:57.131 INFO:teuthology.orchestra.run.smithi097.stdout:(135/152): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-04-25T16:57:57.156 INFO:teuthology.orchestra.run.smithi097.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-04-25T16:57:57.177 INFO:teuthology.orchestra.run.smithi096.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-04-25T16:57:57.181 INFO:teuthology.orchestra.run.smithi097.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-04-25T16:57:57.202 INFO:teuthology.orchestra.run.smithi096.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-04-25T16:57:57.206 INFO:teuthology.orchestra.run.smithi097.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 670 kB/s | 90 kB 00:00 2024-04-25T16:57:57.232 INFO:teuthology.orchestra.run.smithi097.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T16:57:57.257 INFO:teuthology.orchestra.run.smithi097.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T16:57:57.278 INFO:teuthology.orchestra.run.smithi096.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-04-25T16:57:57.282 INFO:teuthology.orchestra.run.smithi097.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-25T16:57:57.303 INFO:teuthology.orchestra.run.smithi096.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-04-25T16:57:57.308 INFO:teuthology.orchestra.run.smithi097.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-25T16:57:57.333 INFO:teuthology.orchestra.run.smithi097.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-25T16:57:57.358 INFO:teuthology.orchestra.run.smithi097.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T16:57:57.383 INFO:teuthology.orchestra.run.smithi097.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-25T16:57:57.393 INFO:teuthology.orchestra.run.smithi096.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T16:57:57.412 INFO:teuthology.orchestra.run.smithi096.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-25T16:57:57.417 INFO:teuthology.orchestra.run.smithi097.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T16:57:57.442 INFO:teuthology.orchestra.run.smithi097.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T16:57:57.487 INFO:teuthology.orchestra.run.smithi096.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 959 kB/s | 90 kB 00:00 2024-04-25T16:57:57.529 INFO:teuthology.orchestra.run.smithi096.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-25T16:57:57.576 INFO:teuthology.orchestra.run.smithi097.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-04-25T16:57:57.579 INFO:teuthology.orchestra.run.smithi096.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-04-25T16:57:57.621 INFO:teuthology.orchestra.run.smithi096.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-25T16:57:57.643 INFO:teuthology.orchestra.run.smithi097.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T16:57:57.688 INFO:teuthology.orchestra.run.smithi096.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-25T16:57:57.730 INFO:teuthology.orchestra.run.smithi096.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-25T16:57:57.760 INFO:teuthology.orchestra.run.smithi097.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T16:57:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-04-25T16:57:57.831 INFO:teuthology.orchestra.run.smithi096.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T16:57:57.881 INFO:teuthology.orchestra.run.smithi096.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 975 kB/s | 90 kB 00:00 2024-04-25T16:57:57.931 INFO:teuthology.orchestra.run.smithi096.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-25T16:57:57.965 INFO:teuthology.orchestra.run.smithi096.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T16:57:58.032 INFO:teuthology.orchestra.run.smithi096.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T16:57:58.598 INFO:teuthology.orchestra.run.smithi096.stdout:(147/152): ceph-test-19.0.0-3291.g994e2f20.el9. 3.7 MB/s | 43 MB 00:11 2024-04-25T16:57:58.696 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-04-25T16:57:58.697 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-04-25T16:57:58.793 INFO:teuthology.orchestra.run.smithi097.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 2.3 MB/s | 3.1 MB 00:01 2024-04-25T16:57:58.949 INFO:teuthology.orchestra.run.smithi096.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-04-25T16:57:58.968 INFO:teuthology.orchestra.run.smithi097.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 2.1 MB/s | 3.3 MB 00:01 2024-04-25T16:57:58.972 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:57:58.972 INFO:teuthology.orchestra.run.smithi097.stdout:Total 11 MB/s | 214 MB 00:19 2024-04-25T16:57:59.199 INFO:teuthology.orchestra.run.smithi096.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 1.3 MB 00:00 2024-04-25T16:57:59.283 INFO:teuthology.orchestra.run.smithi096.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-25T16:57:59.508 INFO:teuthology.orchestra.run.smithi096.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-04-25T16:57:59.650 INFO:teuthology.orchestra.run.smithi096.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 2.1 MB/s | 3.3 MB 00:01 2024-04-25T16:57:59.655 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:57:59.655 INFO:teuthology.orchestra.run.smithi096.stdout:Total 11 MB/s | 214 MB 00:19 2024-04-25T16:58:00.054 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-04-25T16:58:00.197 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-04-25T16:58:00.197 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-04-25T16:58:00.761 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-25T16:58:00.908 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-25T16:58:00.908 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-25T16:58:01.966 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-04-25T16:58:01.967 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-04-25T16:58:02.631 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-25T16:58:02.631 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-25T16:58:04.885 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-04-25T16:58:04.973 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:58:05.012 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:58:05.065 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:58:05.099 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:58:05.125 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:58:05.167 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:58:05.508 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:58:05.626 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:05.752 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:05.777 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:05.870 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:05.897 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:58:06.049 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:58:06.094 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:58:06.123 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:06.219 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:06.249 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:58:06.311 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:58:06.333 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:58:06.356 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:58:06.525 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:58:06.558 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:58:06.665 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:58:06.761 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:58:06.821 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:58:06.833 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:58:06.906 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:58:07.127 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:58:07.201 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:58:07.233 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:58:07.261 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:58:07.329 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:58:07.391 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:58:07.430 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:58:07.465 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:58:07.542 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:58:07.576 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:58:07.620 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:58:07.648 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:58:07.718 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:58:07.763 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:58:07.838 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:58:07.896 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-04-25T16:58:07.970 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:58:07.981 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:58:08.019 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:58:08.024 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:58:08.081 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:58:08.115 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:58:08.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:58:08.175 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:58:08.516 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:58:08.551 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:58:08.650 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:08.653 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-25T16:58:08.729 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:58:08.744 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:58:08.757 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:58:08.776 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:08.783 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:58:08.809 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:08.816 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:58:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:58:08.854 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:58:08.870 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:58:08.895 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:58:08.896 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:58:08.910 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:58:08.939 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:58:09.044 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:58:09.088 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:58:09.090 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:58:09.136 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:58:09.145 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:58:09.164 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:09.220 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:58:09.249 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:58:09.252 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:58:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:58:09.286 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:58:09.327 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:58:09.332 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:58:09.349 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:58:09.368 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:58:09.370 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:58:09.379 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:09.412 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:58:09.454 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:58:09.513 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:09.548 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:58:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:58:09.590 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:58:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:09.635 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:58:09.660 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:58:09.665 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:58:09.697 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:58:09.707 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:58:09.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:58:09.801 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:58:09.809 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:58:09.839 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:58:09.856 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:58:09.865 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:58:09.873 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:58:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:09.905 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:58:09.937 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:58:09.947 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:58:09.972 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:58:09.973 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:58:09.988 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:10.010 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:58:10.016 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:58:10.046 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:58:10.048 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:58:10.073 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:58:10.076 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:58:10.094 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:58:10.117 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:58:10.144 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:58:10.173 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:58:10.198 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:58:10.213 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:58:10.229 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:58:10.241 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:58:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:58:10.265 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:58:10.275 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:58:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:58:10.314 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:58:10.340 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:58:10.346 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:58:10.375 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:58:10.376 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:58:10.401 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:58:10.417 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:58:10.419 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:58:10.444 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:58:10.448 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:58:10.485 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:58:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:58:10.511 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:58:10.515 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:58:10.551 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:58:10.562 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:58:10.586 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:58:10.597 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:58:10.635 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:58:10.642 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:58:10.650 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:58:10.673 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:58:10.685 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:58:10.720 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:58:10.729 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:58:10.761 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:58:10.767 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:58:10.789 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:58:10.820 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:58:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:58:10.857 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:58:10.874 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:58:10.910 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:58:10.941 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:58:10.977 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:58:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:58:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:58:11.131 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:58:11.160 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:58:11.221 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:58:11.274 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:58:11.348 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:58:11.349 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:58:11.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:58:11.457 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:58:11.479 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:58:11.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:58:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:58:11.537 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:58:11.578 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:58:11.608 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:58:11.655 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:58:11.689 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:58:11.725 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:58:11.788 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:58:11.806 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:58:11.839 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:58:11.861 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:58:11.926 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:58:11.934 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:58:11.965 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:58:12.001 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:58:12.008 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:58:12.048 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:58:12.059 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:58:12.084 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:58:12.127 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:58:12.140 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:58:12.169 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:58:12.170 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:58:12.195 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:58:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:58:12.222 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:58:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:58:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:58:12.300 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:58:12.305 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:58:12.350 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:58:12.380 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:58:12.421 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:58:12.454 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:58:12.510 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:58:12.514 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:58:12.547 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:58:12.565 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:58:12.586 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:58:12.612 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:58:12.637 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:58:12.638 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:58:12.669 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:58:12.706 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:58:12.712 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:58:12.752 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:58:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:58:12.829 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:58:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:58:12.861 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:58:12.882 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:58:12.889 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:58:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:58:12.947 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:58:12.962 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:58:12.993 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:58:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:58:13.023 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:58:13.058 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:58:13.062 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:58:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:58:13.092 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:58:13.100 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:58:13.133 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:58:13.141 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:58:13.167 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:58:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:58:13.253 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:58:13.275 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:58:13.297 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:58:13.315 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:58:13.329 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:58:13.375 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:58:13.473 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:58:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:58:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:58:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:58:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:58:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:58:14.031 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:58:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:58:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:58:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:58:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:58:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:58:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:58:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:58:14.290 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:58:14.295 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:58:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:58:14.360 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:58:14.385 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:58:14.411 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:58:14.416 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:58:14.421 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:58:14.444 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:58:14.454 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:58:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:58:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:58:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:58:14.542 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:58:14.594 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:58:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:58:14.673 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:58:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:58:14.713 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:58:14.776 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:58:14.817 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:58:14.891 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:58:14.892 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:58:14.922 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:58:14.927 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:58:14.958 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:58:15.038 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:58:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:58:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:58:15.449 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:58:15.478 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:58:15.481 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:58:15.549 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:58:15.571 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:58:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:58:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:58:15.686 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:58:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:58:15.758 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:58:15.799 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:58:15.936 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:58:15.964 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:58:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:58:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:58:16.787 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:58:17.026 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:58:17.107 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:58:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:58:17.167 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:58:17.490 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:58:17.530 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:58:17.629 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:58:17.660 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:58:17.695 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:58:17.816 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:17.910 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:18.008 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:58:18.085 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:58:18.112 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:58:18.146 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:58:18.185 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:58:18.208 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:58:18.279 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:58:18.301 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:58:18.467 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:58:18.468 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:18.505 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:58:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:58:18.594 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:58:18.594 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:58:18.595 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:58:18.595 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:18.599 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:58:18.623 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:58:18.630 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:58:18.666 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:58:19.154 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:58:19.178 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:58:19.249 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:58:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:58:20.548 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:20.581 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:21.140 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:21.176 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:58:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:58:21.267 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:58:21.267 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:58:21.267 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:21.293 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:58:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:58:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:58:22.214 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:58:22.214 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:58:22.214 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:58:22.214 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:22.247 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:58:30.520 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:58:30.520 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-04-25T16:58:30.520 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-04-25T16:58:30.520 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-04-25T16:58:30.520 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-04-25T16:58:30.520 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-04-25T16:58:30.520 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-04-25T16:58:30.521 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-04-25T16:58:30.521 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:58:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:58:30.700 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:30.700 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:58:30.700 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:32.272 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:58:32.298 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:58:32.298 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:32.298 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:58:32.298 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:58:32.298 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:58:32.298 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:32.523 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:58:32.545 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:58:32.545 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:32.545 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:58:32.545 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:58:32.546 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:58:32.546 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:32.788 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-04-25T16:58:32.789 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:32.964 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:58:32.999 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:58:32.999 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:32.999 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:58:32.999 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:33.007 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:58:33.030 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:58:33.030 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:33.030 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:58:33.030 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:58:33.030 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:58:33.030 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:33.129 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:58:33.185 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:58:33.250 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:58:33.435 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:58:33.504 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:58:33.577 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:58:33.666 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:58:33.814 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:58:33.858 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:58:33.890 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:58:33.917 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:58:33.990 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:58:33.990 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-04-25T16:58:33.991 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-04-25T16:58:33.991 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-04-25T16:58:33.991 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-04-25T16:58:33.991 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-04-25T16:58:33.991 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-04-25T16:58:33.991 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-04-25T16:58:33.991 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:34.158 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:58:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:58:34.174 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:34.174 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:58:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:34.352 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:58:34.382 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:58:34.414 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:58:34.477 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:58:34.506 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:58:34.550 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:58:34.575 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:58:34.575 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:34.575 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:58:34.575 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:58:34.575 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:58:34.575 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:34.800 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:58:34.822 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:58:34.822 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:34.822 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:58:34.822 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:58:34.822 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:58:34.822 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:35.285 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:58:35.306 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:58:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:58:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:58:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:58:35.307 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:35.406 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:58:35.454 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:58:35.509 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:58:35.541 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:58:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:58:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:58:35.743 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:58:35.752 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:58:35.777 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:58:35.778 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:35.778 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:58:35.778 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:58:35.778 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:58:35.778 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:35.820 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:58:35.908 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:58:36.003 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:58:36.025 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:58:36.025 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:36.025 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:58:36.025 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:58:36.025 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:58:36.025 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:36.046 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:58:36.085 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:58:36.117 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:58:36.170 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:58:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:58:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:58:36.509 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:36.509 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:58:36.509 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:58:36.510 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:58:36.510 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:36.562 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:58:36.571 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:58:36.591 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:58:36.602 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:58:36.608 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:58:36.634 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:58:36.657 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:58:36.693 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:58:36.694 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:58:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:58:36.733 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:58:36.867 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:58:36.893 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:58:36.906 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:58:36.917 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:36.954 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:58:36.961 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:58:36.972 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:58:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:58:37.099 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:58:37.119 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:58:37.151 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:58:37.257 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:58:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:58:37.328 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:58:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:58:37.759 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:58:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:58:37.793 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:58:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:58:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:58:37.898 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:58:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:58:38.082 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:58:38.113 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:58:38.136 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:58:38.136 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:38.136 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:58:38.137 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:58:38.137 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:58:38.137 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:38.189 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:58:38.211 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:58:38.211 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:38.212 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:58:38.212 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:38.256 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:58:38.279 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:58:38.279 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:38.279 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:58:38.279 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:58:38.280 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:58:38.280 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:38.591 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:58:38.613 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:58:38.613 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:38.613 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:58:38.613 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:58:38.613 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:58:38.614 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:38.754 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:58:38.789 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:58:38.890 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:58:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:58:39.012 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:58:39.059 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:58:39.087 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:58:39.110 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:58:39.110 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:39.110 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:58:39.110 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:58:39.110 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:58:39.110 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:39.155 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:58:39.165 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:58:39.265 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:58:39.311 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:58:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:58:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:58:40.114 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:58:40.245 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:58:40.271 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:58:40.282 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:58:40.304 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:58:40.305 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:40.305 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:58:40.305 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:58:40.305 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:58:40.305 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:40.315 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:58:40.337 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:58:40.337 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:40.337 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:58:40.337 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:58:40.337 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:58:40.338 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:40.349 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:58:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:58:40.371 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:40.371 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:58:40.371 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:40.390 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:58:40.402 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:58:40.416 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:58:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:58:40.439 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:40.439 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:58:40.439 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:58:40.439 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:58:40.439 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:40.501 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:58:40.555 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:58:40.759 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:58:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:58:40.781 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:40.781 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:58:40.781 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:58:40.781 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:58:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:41.492 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:58:41.526 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:58:41.548 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:58:41.549 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:41.549 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:58:41.549 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:58:41.549 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:58:41.549 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:41.594 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:58:41.615 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:58:41.615 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:41.615 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:58:41.615 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:41.660 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:58:41.682 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:58:41.682 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:41.682 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:58:41.682 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:58:41.682 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:58:41.682 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:42.002 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:58:42.025 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:58:42.025 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:58:42.025 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:58:42.025 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:58:42.025 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:58:42.025 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:42.894 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:58:42.936 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:58:42.976 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:58:43.008 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:58:43.044 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:58:43.218 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:58:43.265 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:58:43.295 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:58:43.908 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:43.943 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:43.962 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:44.798 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:58:44.881 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:58:44.921 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:58:45.006 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:58:45.045 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:58:45.045 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:58:45.063 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:58:45.104 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:58:45.112 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:58:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:58:45.137 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:58:45.144 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:58:45.176 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:58:45.203 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:58:45.216 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:58:45.285 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:58:45.433 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:58:45.463 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:58:46.016 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:46.052 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:46.072 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:46.273 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:58:46.314 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:58:46.354 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:58:46.386 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:58:46.413 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:58:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:58:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:58:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:58:46.897 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:58:46.973 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:58:47.014 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:58:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:58:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:58:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:58:47.211 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:58:47.236 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:47.244 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:58:47.244 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:58:47.279 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:47.299 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:58:47.323 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:58:47.395 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:48.124 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:58:48.217 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:58:48.257 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:58:48.340 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:58:48.370 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:58:48.370 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:58:48.471 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:58:48.513 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:58:48.513 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:58:48.606 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:58:48.678 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:51.625 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:58:51.626 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:58:51.627 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:58:51.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:58:51.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:58:51.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:58:51.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:58:51.628 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:58:51.640 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:58:51.640 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:58:51.641 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:58:51.642 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:58:51.643 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:58:51.644 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:58:51.645 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:58:51.646 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:58:51.647 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:58:51.648 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:58:51.649 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:58:51.650 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:58:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:58:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:58:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:58:53.236 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:58:53.237 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:58:53.238 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:58:53.239 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:58:53.240 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:58:53.241 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:58:53.242 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:58:53.243 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:58:53.244 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:58:53.245 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:58:53.246 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:58:53.247 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:58:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:53.669 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-04-25T16:58:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.669 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.670 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:53.671 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.672 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:58:53.673 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:58:53.674 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:58:53.675 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:58:53.676 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:58:53.677 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:58:53.678 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:58:53.679 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:58:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:58:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:58:53.680 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T16:58:53.680 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-25T16:58:54.076 DEBUG:teuthology.parallel:result is None 2024-04-25T16:58:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:54.724 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:58:54.725 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:58:54.726 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:58:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:58:54.727 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:58:54.739 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:58:54.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:58:54.741 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:58:54.742 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:58:54.743 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:58:54.744 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:58:54.745 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:58:54.746 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:58:54.747 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:58:54.748 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:55.331 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.332 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.333 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.334 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:58:55.335 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:58:55.336 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:58:55.337 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:58:55.338 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:58:55.339 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:58:55.340 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:58:55.341 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:58:55.342 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:58:55.343 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:58:55.343 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T16:58:55.343 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-25T16:58:55.709 DEBUG:teuthology.parallel:result is None 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-25T16:58:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:58:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:58:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:58:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:58:56.790 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:58:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:58:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:58:56.793 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:58:56.794 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:58:56.795 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T16:58:56.795 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T16:58:57.187 DEBUG:teuthology.parallel:result is None 2024-04-25T16:58:57.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:58:57.325 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:58:57.346 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:58:57.347 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:58:57.347 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:58:57.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:58:57.560 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:58:57.581 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:58:57.581 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:58:57.581 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:58:57.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:58:57.793 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:58:57.819 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:58:57.819 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:58:57.820 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:58:57.821 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T16:58:57.822 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T16:58:57.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:58:57.849 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T16:58:57.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:58:57.875 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T16:58:57.875 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:58:57.905 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T16:58:57.905 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T16:58:57.905 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:58:57.930 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:58:57.995 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T16:58:57.995 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:58:58.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:58:58.081 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T16:58:58.081 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:58:58.109 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:58:58.175 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T16:58:58.176 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T16:58:58.176 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:58:58.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:58:58.266 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T16:58:58.266 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:58:58.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:58:58.353 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T16:58:58.353 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:58:58.380 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:58:58.447 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T16:58:58.447 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T16:58:58.448 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:58:58.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:58:58.534 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T16:58:58.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:58:58.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:58:58.621 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T16:58:58.621 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:58:58.648 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:58:58.713 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T16:58:58.814 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T16:58:58.814 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:58:58.815 INFO:tasks.cephadm:Cluster fsid is 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T16:58:58.815 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T16:58:58.815 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.96', 'mon.b': '172.21.15.97', 'mon.c': '172.21.15.171'} 2024-04-25T16:58:58.816 INFO:tasks.cephadm:First mon is mon.a on smithi096 2024-04-25T16:58:58.816 INFO:tasks.cephadm:First mgr is x 2024-04-25T16:58:58.816 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T16:58:58.816 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-04-25T16:58:58.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-04-25T16:58:58.863 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-04-25T16:58:58.889 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T16:58:58.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:58:59.103 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T16:58:59.315 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:58:59.317 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T16:58:59.317 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:00.353 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 16:59 /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:00.354 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:01.419 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 16:59 /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:01.419 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:02.530 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 16:59 /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:02.531 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:02.549 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:02.568 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T16:59:02.603 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T16:59:02.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T16:59:02.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T16:59:02.610 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T16:59:04.163 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:59:04.290 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:59:04.316 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:59:37.615 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T16:59:37.615 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:59:37.615 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T16:59:37.615 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-04-25T16:59:37.616 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T16:59:37.616 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-04-25T16:59:37.616 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T16:59:56.791 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T16:59:56.791 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:59:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T16:59:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-04-25T16:59:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T16:59:56.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T16:59:56.792 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T16:59:59.167 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T16:59:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:59:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T16:59:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-04-25T16:59:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T16:59:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T16:59:59.168 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T16:59:59.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-04-25T16:59:59.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-04-25T16:59:59.249 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-04-25T16:59:59.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-04-25T16:59:59.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-04-25T16:59:59.330 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-04-25T16:59:59.358 INFO:tasks.cephadm:Writing seed config... 2024-04-25T16:59:59.359 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T16:59:59.359 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T16:59:59.359 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T16:59:59.359 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T16:59:59.360 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T16:59:59.361 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T16:59:59.362 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T16:59:59.362 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T16:59:59.362 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T16:59:59.362 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T16:59:59.362 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T16:59:59.362 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T16:59:59.362 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T16:59:59.363 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T16:59:59.363 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T16:59:59.380 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1be09d02-0325-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T16:59:59.380 DEBUG:teuthology.orchestra.run.smithi096:mon.a> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T16:59:59.423 DEBUG:teuthology.orchestra.run.smithi096:mgr.x> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T16:59:59.465 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T16:59:59.465 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid 1be09d02-0325-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.96 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:59:59.657 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:59:59.657 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', '1be09d02-0325-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.96', '--skip-admin-label'] 2024-04-25T16:59:59.658 INFO:teuthology.orchestra.run.smithi096.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T16:59:59.658 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying podman|docker is present... 2024-04-25T16:59:59.696 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T16:59:59.696 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying lvm2 is present... 2024-04-25T16:59:59.697 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying time synchronization is in place... 2024-04-25T16:59:59.704 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T16:59:59.704 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T16:59:59.710 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T16:59:59.710 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T16:59:59.716 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-04-25T16:59:59.722 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-04-25T16:59:59.722 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-04-25T16:59:59.722 INFO:teuthology.orchestra.run.smithi096.stdout:Repeating the final host check... 2024-04-25T16:59:59.746 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T16:59:59.746 INFO:teuthology.orchestra.run.smithi096.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T16:59:59.746 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl is present 2024-04-25T16:59:59.746 INFO:teuthology.orchestra.run.smithi096.stdout:lvcreate is present 2024-04-25T16:59:59.752 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T16:59:59.752 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T16:59:59.758 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T16:59:59.758 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T16:59:59.765 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-04-25T16:59:59.771 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-04-25T16:59:59.771 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-04-25T16:59:59.771 INFO:teuthology.orchestra.run.smithi096.stdout:Host looks OK 2024-04-25T16:59:59.771 INFO:teuthology.orchestra.run.smithi096.stdout:Cluster fsid: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T16:59:59.771 INFO:teuthology.orchestra.run.smithi096.stdout:Acquiring lock 140628330819536 on /run/cephadm/1be09d02-0325-11ef-bc93-c7b262605968.lock 2024-04-25T16:59:59.771 INFO:teuthology.orchestra.run.smithi096.stdout:Lock 140628330819536 acquired on /run/cephadm/1be09d02-0325-11ef-bc93-c7b262605968.lock 2024-04-25T16:59:59.772 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 3300 ... 2024-04-25T16:59:59.772 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 6789 ... 2024-04-25T16:59:59.772 INFO:teuthology.orchestra.run.smithi096.stdout:Base mon IP(s) is [172.21.15.96:3300, 172.21.15.96:6789], mon addrv is [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-04-25T16:59:59.775 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.96 metric 100 2024-04-25T16:59:59.775 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.96 metric 100 2024-04-25T16:59:59.778 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T16:59:59.778 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T16:59:59.778 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T16:59:59.780 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T16:59:59.780 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T16:59:59.780 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T16:59:59.780 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T16:59:59.780 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 fe80::ddd4:c54b:dd9f:768b/64 scope link noprefixroute 2024-04-25T16:59:59.780 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T16:59:59.781 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-04-25T16:59:59.781 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-04-25T16:59:59.781 INFO:teuthology.orchestra.run.smithi096.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T16:59:59.781 INFO:teuthology.orchestra.run.smithi096.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T16:59:59.782 INFO:teuthology.orchestra.run.smithi096.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T17:00:00.269 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T17:00:00.270 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T17:00:00.270 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T17:00:00.270 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T17:00:00.270 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T17:00:00.270 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T17:00:00.270 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T17:00:01.266 INFO:teuthology.orchestra.run.smithi096.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T17:00:01.266 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T17:00:01.266 INFO:teuthology.orchestra.run.smithi096.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T17:00:02.178 INFO:teuthology.orchestra.run.smithi096.stdout:stat: stdout 167 167 2024-04-25T17:00:02.178 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial keys... 2024-04-25T17:00:03.092 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQASjCpm6KH7HhAAafYE6QszqZ2ZuIlIDL6kjA== 2024-04-25T17:00:03.990 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQATjCpmsdBQGRAA70Fayi5Ew8O1kcdeX1kscQ== 2024-04-25T17:00:04.997 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQAUjCpm1tTYFhAA4LyJ1l9fPsguMmXK0G/t8g== 2024-04-25T17:00:04.998 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial monmap... 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool for a [v2:172.21.15.96:3300,v1:172.21.15.96:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: set fsid to 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:00:06.145 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mon... 2024-04-25T17:00:07.042 INFO:teuthology.orchestra.run.smithi096.stdout:create mon.a on 2024-04-25T17:00:07.240 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T17:00:07.377 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T17:00:07.551 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1be09d02-0325-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-1be09d02-0325-11ef-bc93-c7b262605968.target. 2024-04-25T17:00:07.551 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1be09d02-0325-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-1be09d02-0325-11ef-bc93-c7b262605968.target. 2024-04-25T17:00:07.767 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a 2024-04-25T17:00:07.767 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T17:00:07.942 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1be09d02-0325-11ef-bc93-c7b262605968.target.wants/ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1be09d02-0325-11ef-bc93-c7b262605968@.service. 2024-04-25T17:00:08.477 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T17:00:08.477 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T17:00:08.484 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T17:00:08.485 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T17:00:08.485 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T17:00:08.485 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T17:00:08.485 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon to start... 2024-04-25T17:00:08.485 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon... 2024-04-25T17:00:09.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T17:00:09.679 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout id: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout services: 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.630904s) 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout data: 2024-04-25T17:00:09.680 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.964+0000 7f0504347700 1 Processor -- start 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.964+0000 7f0504347700 1 -- start start 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.964+0000 7f0504347700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc1086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.964+0000 7f0504347700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc07d830 con 0x7f04fc108290 2024-04-25T17:00:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.964+0000 7f05020e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc1086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.965+0000 7f05020e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc1086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54786/0 (socket says 172.21.15.96:54786) 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.965+0000 7f05020e3700 1 -- 172.21.15.96:0/582338523 learned_addr learned my addr 172.21.15.96:0/582338523 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.965+0000 7f05020e3700 1 -- 172.21.15.96:0/582338523 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc108be0 con 0x7f04fc108290 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.965+0000 7f05020e3700 1 --2- 172.21.15.96:0/582338523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc1086a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04f80098a0 tx=0x7f04f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e465f52fb1f3eaa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.965+0000 7f05010e1700 1 -- 172.21.15.96:0/582338523 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f8004160 con 0x7f04fc108290 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.965+0000 7f05010e1700 1 -- 172.21.15.96:0/582338523 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8004300 con 0x7f04fc108290 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.965+0000 7f05010e1700 1 -- 172.21.15.96:0/582338523 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f80036a0 con 0x7f04fc108290 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.966+0000 7f0504347700 1 -- 172.21.15.96:0/582338523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 msgr2=0x7f04fc1086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:09.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.966+0000 7f0504347700 1 --2- 172.21.15.96:0/582338523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc1086a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f04f80098a0 tx=0x7f04f8009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.966+0000 7f0504347700 1 -- 172.21.15.96:0/582338523 shutdown_connections 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.966+0000 7f0504347700 1 --2- 172.21.15.96:0/582338523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc1086a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.966+0000 7f0504347700 1 -- 172.21.15.96:0/582338523 >> 172.21.15.96:0/582338523 conn(0x7f04fc103e80 msgr2=0x7f04fc106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.966+0000 7f0504347700 1 -- 172.21.15.96:0/582338523 shutdown_connections 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.966+0000 7f0504347700 1 -- 172.21.15.96:0/582338523 wait complete. 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.967+0000 7f0504347700 1 Processor -- start 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.967+0000 7f0504347700 1 -- start start 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.967+0000 7f0504347700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:09.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.967+0000 7f0504347700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc109110 con 0x7f04fc108290 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.967+0000 7f05020e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.967+0000 7f05020e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54790/0 (socket says 172.21.15.96:54790) 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.967+0000 7f05020e3700 1 -- 172.21.15.96:0/3167808061 learned_addr learned my addr 172.21.15.96:0/3167808061 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.968+0000 7f05020e3700 1 -- 172.21.15.96:0/3167808061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc19fc40 con 0x7f04fc108290 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.968+0000 7f05020e3700 1 --2- 172.21.15.96:0/3167808061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc19f700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f04f8000c00 tx=0x7f04f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.968+0000 7f04f37fe700 1 -- 172.21.15.96:0/3167808061 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f8003db0 con 0x7f04fc108290 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.968+0000 7f04f37fe700 1 -- 172.21.15.96:0/3167808061 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8011630 con 0x7f04fc108290 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.968+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc19fed0 con 0x7f04fc108290 2024-04-25T17:00:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.968+0000 7f04f37fe700 1 -- 172.21.15.96:0/3167808061 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04f8011950 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.968+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1a2ae0 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.969+0000 7f04f37fe700 1 -- 172.21.15.96:0/3167808061 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f04f801a460 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.969+0000 7f04f37fe700 1 -- 172.21.15.96:0/3167808061 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f04f801b780 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.969+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc064cf0 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:08.974+0000 7f04f37fe700 1 -- 172.21.15.96:0/3167808061 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f04f8031020 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.057+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f04fc1a2cb0 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.058+0000 7f04f37fe700 1 -- 172.21.15.96:0/3167808061 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f04f801b9d0 con 0x7f04fc108290 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.059+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 msgr2=0x7f04fc19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.059+0000 7f0504347700 1 --2- 172.21.15.96:0/3167808061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc19f700 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f04f8000c00 tx=0x7f04f8004750 comp rx=0 tx=0).stop 2024-04-25T17:00:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.059+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 shutdown_connections 2024-04-25T17:00:09.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.059+0000 7f0504347700 1 --2- 172.21.15.96:0/3167808061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04fc108290 0x7f04fc19f700 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:09.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.059+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 >> 172.21.15.96:0/3167808061 conn(0x7f04fc103e80 msgr2=0x7f04fc1959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:09.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.059+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 shutdown_connections 2024-04-25T17:00:09.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:09.059+0000 7f0504347700 1 -- 172.21.15.96:0/3167808061 wait complete. 2024-04-25T17:00:09.686 INFO:teuthology.orchestra.run.smithi096.stdout:mon is available 2024-04-25T17:00:09.686 INFO:teuthology.orchestra.run.smithi096.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T17:00:10.674 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:10 smithi096 ceph-mon[28747]: from='client.? 172.21.15.96:0/3932493957' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T17:00:10.682 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T17:00:10.683 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.038+0000 7f2353823700 1 Processor -- start 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.038+0000 7f2353823700 1 -- start start 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.038+0000 7f2353823700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c10c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.038+0000 7f2353823700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f234c07d710 con 0x7f234c10a450 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.038+0000 7f23515bf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c10c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.038+0000 7f23515bf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c10c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54794/0 (socket says 172.21.15.96:54794) 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.038+0000 7f23515bf700 1 -- 172.21.15.96:0/3021757137 learned_addr learned my addr 172.21.15.96:0/3021757137 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.039+0000 7f23515bf700 1 -- 172.21.15.96:0/3021757137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c10cdb0 con 0x7f234c10a450 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.039+0000 7f23515bf700 1 --2- 172.21.15.96:0/3021757137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c10c870 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f233c0098a0 tx=0x7f233c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918de772b076df71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.039+0000 7f2343fff700 1 -- 172.21.15.96:0/3021757137 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c004160 con 0x7f234c10a450 2024-04-25T17:00:10.684 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.039+0000 7f2343fff700 1 -- 172.21.15.96:0/3021757137 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f233c004300 con 0x7f234c10a450 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.039+0000 7f2343fff700 1 -- 172.21.15.96:0/3021757137 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c0036a0 con 0x7f234c10a450 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 -- 172.21.15.96:0/3021757137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 msgr2=0x7f234c10c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 --2- 172.21.15.96:0/3021757137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c10c870 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f233c0098a0 tx=0x7f233c009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 -- 172.21.15.96:0/3021757137 shutdown_connections 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 --2- 172.21.15.96:0/3021757137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c10c870 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 -- 172.21.15.96:0/3021757137 >> 172.21.15.96:0/3021757137 conn(0x7f234c103ed0 msgr2=0x7f234c106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 -- 172.21.15.96:0/3021757137 shutdown_connections 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 -- 172.21.15.96:0/3021757137 wait complete. 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 Processor -- start 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.040+0000 7f2353823700 1 -- start start 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f2353823700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f2353823700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f234c10d220 con 0x7f234c10a450 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23515bf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:10.685 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23515bf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c19d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54800/0 (socket says 172.21.15.96:54800) 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23515bf700 1 -- 172.21.15.96:0/3932493957 learned_addr learned my addr 172.21.15.96:0/3932493957 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23515bf700 1 -- 172.21.15.96:0/3932493957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c19da40 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23515bf700 1 --2- 172.21.15.96:0/3932493957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c19d500 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f233c000c00 tx=0x7f233c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c003db0 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f233c011630 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f233c011950 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.041+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c19dcd0 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.042+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c1a08e0 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.042+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f233c01a460 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.042+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f233c01b780 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.043+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c064cf0 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.045+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f233c031020 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.111+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f234c02e7e0 con 0x7f234c10a450 2024-04-25T17:00:10.686 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.115+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f233c01bad0 con 0x7f234c10a450 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.116+0000 7f23427fc700 1 -- 172.21.15.96:0/3932493957 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f233c01ad70 con 0x7f234c10a450 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.117+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 msgr2=0x7f234c19d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.117+0000 7f2353823700 1 --2- 172.21.15.96:0/3932493957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c19d500 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f233c000c00 tx=0x7f233c004750 comp rx=0 tx=0).stop 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.117+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 shutdown_connections 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.117+0000 7f2353823700 1 --2- 172.21.15.96:0/3932493957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f234c10a450 0x7f234c19d500 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.117+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 >> 172.21.15.96:0/3932493957 conn(0x7f234c103ed0 msgr2=0x7f234c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.117+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 shutdown_connections 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:10.117+0000 7f2353823700 1 -- 172.21.15.96:0/3932493957 wait complete. 2024-04-25T17:00:10.687 INFO:teuthology.orchestra.run.smithi096.stdout:Generating new minimal ceph.conf... 2024-04-25T17:00:11.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:11 smithi096 ceph-mon[28747]: from='client.? 172.21.15.96:0/351671545' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:00:11.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9aed8ae700 1 Processor -- start 2024-04-25T17:00:11.779 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9aed8ae700 1 -- start start 2024-04-25T17:00:11.779 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9aed8ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae8108c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9aed8ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8109270 con 0x7f9ae8108870 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae8108c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae8108c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54808/0 (socket says 172.21.15.96:54808) 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9ae6ffd700 1 -- 172.21.15.96:0/2397704794 learned_addr learned my addr 172.21.15.96:0/2397704794 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9ae6ffd700 1 -- 172.21.15.96:0/2397704794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae8109a40 con 0x7f9ae8108870 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.097+0000 7f9ae6ffd700 1 --2- 172.21.15.96:0/2397704794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae8108c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098a0 tx=0x7f9ad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35f1a2e03e5e07a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9ae5ffb700 1 -- 172.21.15.96:0/2397704794 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ad8004160 con 0x7f9ae8108870 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9ae5ffb700 1 -- 172.21.15.96:0/2397704794 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9ad8004300 con 0x7f9ae8108870 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9ae5ffb700 1 -- 172.21.15.96:0/2397704794 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ad8011af0 con 0x7f9ae8108870 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9aed8ae700 1 -- 172.21.15.96:0/2397704794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 msgr2=0x7f9ae8108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9aed8ae700 1 --2- 172.21.15.96:0/2397704794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae8108c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80098a0 tx=0x7f9ad8009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9aed8ae700 1 -- 172.21.15.96:0/2397704794 shutdown_connections 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9aed8ae700 1 --2- 172.21.15.96:0/2397704794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae8108c80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:11.780 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9aed8ae700 1 -- 172.21.15.96:0/2397704794 >> 172.21.15.96:0/2397704794 conn(0x7f9ae8103f50 msgr2=0x7f9ae81063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9aed8ae700 1 -- 172.21.15.96:0/2397704794 shutdown_connections 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.098+0000 7f9aed8ae700 1 -- 172.21.15.96:0/2397704794 wait complete. 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.099+0000 7f9aed8ae700 1 Processor -- start 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.099+0000 7f9aed8ae700 1 -- start start 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.099+0000 7f9aed8ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.099+0000 7f9aed8ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae810cdf0 con 0x7f9ae8108870 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54820/0 (socket says 172.21.15.96:54820) 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9ae6ffd700 1 -- 172.21.15.96:0/351671545 learned_addr learned my addr 172.21.15.96:0/351671545 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9ae6ffd700 1 -- 172.21.15.96:0/351671545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae819d230 con 0x7f9ae8108870 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9ae6ffd700 1 --2- 172.21.15.96:0/351671545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae819ccf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011890 tx=0x7f9ad8011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9aec8ac700 1 -- 172.21.15.96:0/351671545 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ad80039e0 con 0x7f9ae8108870 2024-04-25T17:00:11.781 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9aec8ac700 1 -- 172.21.15.96:0/351671545 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9ad8003b80 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae819d4c0 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9aec8ac700 1 -- 172.21.15.96:0/351671545 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ad80246e0 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae81a00d0 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9aec8ac700 1 -- 172.21.15.96:0/351671545 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9ad8022020 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.100+0000 7f9aec8ac700 1 -- 172.21.15.96:0/351671545 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9ad802dc60 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.101+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae8064cf0 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.103+0000 7f9aec8ac700 1 -- 172.21.15.96:0/351671545 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9ad8043e50 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.170+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9ae81a03c0 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.170+0000 7f9aec8ac700 1 -- 172.21.15.96:0/351671545 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f9ad8032020 con 0x7f9ae8108870 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.171+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 msgr2=0x7f9ae819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.171+0000 7f9aed8ae700 1 --2- 172.21.15.96:0/351671545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae819ccf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011890 tx=0x7f9ad8011970 comp rx=0 tx=0).stop 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.171+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 shutdown_connections 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.171+0000 7f9aed8ae700 1 --2- 172.21.15.96:0/351671545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ae8108870 0x7f9ae819ccf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:11.782 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.171+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 >> 172.21.15.96:0/351671545 conn(0x7f9ae8103f50 msgr2=0x7f9ae8105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:11.783 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.171+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 shutdown_connections 2024-04-25T17:00:11.783 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:11.171+0000 7f9aed8ae700 1 -- 172.21.15.96:0/351671545 wait complete. 2024-04-25T17:00:11.783 INFO:teuthology.orchestra.run.smithi096.stdout:Restarting the monitor... 2024-04-25T17:00:12.068 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:11 smithi096 systemd[1]: Stopping Ceph mon.a for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:00:12.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a[28742]: 2024-04-25T17:00:12.070+0000 7f3287b18700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:00:12.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a[28742]: 2024-04-25T17:00:12.070+0000 7f3287b18700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T17:00:12.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 podman[29004]: 2024-04-25 17:00:12.14213943 +0000 UTC m=+0.186805885 container died d627f8fc9d84d26afe14cb97e21178abf0542118469795f81bcd4c27b594ab61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T17:00:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 podman[29004]: 2024-04-25 17:00:12.46164725 +0000 UTC m=+0.506313703 container cleanup d627f8fc9d84d26afe14cb97e21178abf0542118469795f81bcd4c27b594ab61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T17:00:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 bash[29004]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a 2024-04-25T17:00:13.027 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 podman[29018]: 2024-04-25 17:00:12.696175026 +0000 UTC m=+0.552852210 container remove d627f8fc9d84d26afe14cb97e21178abf0542118469795f81bcd4c27b594ab61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T17:00:13.027 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T17:00:13.027 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 systemd[1]: Stopped Ceph mon.a for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:00:13.027 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:12 smithi096 systemd[1]: Starting Ceph mon.a for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:00:13.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 podman[29112]: 2024-04-25 17:00:13.03041673 +0000 UTC m=+0.093243786 container create 08b03381202958ddfe15893731e59e5c00190c27de01501ad6919006009b6a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:00:13.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 podman[29112]: 2024-04-25 17:00:12.955134955 +0000 UTC m=+0.017961968 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T17:00:13.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 podman[29112]: 2024-04-25 17:00:13.232782017 +0000 UTC m=+0.295609017 container init 08b03381202958ddfe15893731e59e5c00190c27de01501ad6919006009b6a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T17:00:13.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 podman[29112]: 2024-04-25 17:00:13.235616273 +0000 UTC m=+0.298443283 container start 08b03381202958ddfe15893731e59e5c00190c27de01501ad6919006009b6a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:00:13.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: pidfile_write: ignore empty --pid-file 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: load: jerasure load: lrc 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: RocksDB version: 7.9.2 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Git sha 0 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: DB SUMMARY 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: DB Session ID: 0CCD5ZTQ39AGCYNALSPO 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: CURRENT file: CURRENT 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T17:00:13.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104530 ; 2024-04-25T17:00:13.315 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.error_if_exists: 0 2024-04-25T17:00:13.315 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.create_if_missing: 0 2024-04-25T17:00:13.315 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.paranoid_checks: 1 2024-04-25T17:00:13.315 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T17:00:13.315 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.env: 0x55ee6564e500 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.info_log: 0x55ee668b5fc0 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.statistics: (nil) 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.use_fsync: 0 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_log_file_size: 0 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.allow_fallocate: 1 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T17:00:13.316 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.use_direct_reads: 0 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.db_log_dir: 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.wal_dir: 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T17:00:13.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.write_buffer_manager: 0x55ee668868c0 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.unordered_write: 0 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.row_cache: None 2024-04-25T17:00:13.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.wal_filter: None 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.two_write_queues: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.wal_compression: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.atomic_flush: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.log_readahead_size: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T17:00:13.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_background_jobs: 2 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_background_compactions: -1 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_subcompactions: 1 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_open_files: -1 2024-04-25T17:00:13.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_background_flushes: -1 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Compression algorithms supported: 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kZSTD supported: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kXpressCompression supported: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kZlibCompression supported: 1 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kSnappyCompression supported: 1 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kLZ4Compression supported: 1 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: kBZip2Compression supported: 0 2024-04-25T17:00:13.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.merge_operator: 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_filter: None 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ee669bc1a0) 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T17:00:13.322 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_type: 0 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_index_type: 0 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_shortening: 1 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: checksum: 4 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: no_block_cache: 0 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache: 0x55ee668b91f0 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_options: 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: capacity : 536870912 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_shard_bits : 4 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: strict_capacity_limit : 0 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_compressed: (nil) 2024-04-25T17:00:13.323 INFO:journalctl@ceph.mon.a.smithi096.stdout: persistent_cache: (nil) 2024-04-25T17:00:13.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size: 4096 2024-04-25T17:00:13.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size_deviation: 10 2024-04-25T17:00:13.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_restart_interval: 16 2024-04-25T17:00:13.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_block_restart_interval: 1 2024-04-25T17:00:13.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: metadata_block_size: 4096 2024-04-25T17:00:13.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: partition_filters: 0 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: use_delta_encoding: 1 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: filter_policy: bloomfilter 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: whole_key_filtering: 1 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: verify_compression: 0 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: format_version: 5 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_index_compression: 1 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_align: 0 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: max_auto_readahead_size: 262144 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: prepopulate_block_cache: 0 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: initial_auto_readahead_size: 8192 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T17:00:13.325 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression: NoCompression 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.num_levels: 7 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T17:00:13.326 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T17:00:13.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T17:00:13.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T17:00:13.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.inplace_update_support: 0 2024-04-25T17:00:13.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T17:00:13.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T17:00:13.330 INFO:teuthology.orchestra.run.smithi096.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T17:00:13.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.bloom_locality: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.max_successive_merges: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.ttl: 2592000 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.enable_blob_files: false 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.min_blob_size: 0 2024-04-25T17:00:13.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 292b737f-54a5-4ca4-a35a-ac2d52448d48 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714064413274738, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714064413275874, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99757, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97639, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14028, "raw_average_key_size": 49, "raw_value_size": 89997, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714064413, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "292b737f-54a5-4ca4-a35a-ac2d52448d48", "db_session_id": "0CCD5ZTQ39AGCYNALSPO", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714064413275955, "job": 1, "event": "recovery_finished"} 2024-04-25T17:00:13.332 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ee6689dc00 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: DB pointer 0x55ee6b690000 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** DB Stats ** 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T17:00:13.333 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: L0 2/0 99.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: Sum 2/0 99.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T17:00:13.334 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative compaction: 0.00 GB write, 30.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval compaction: 0.00 GB write, 30.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache BinnedLRUCache@0x55ee668b91f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: starting mon.a rank 0 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:13.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???) e1 preinit fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???).mds e1 new map 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???).mds e1 print_map 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout: e1 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout: legacy client fscid: -1 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout: No filesystems configured 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 bash[29112]: 08b03381202958ddfe15893731e59e5c00190c27de01501ad6919006009b6a88 2024-04-25T17:00:13.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 systemd[1]: Started Ceph mon.a for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:00:13.662 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T17:00:13.662 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: monmap epoch 1 2024-04-25T17:00:13.662 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:13.662 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: last_changed 2024-04-25T17:00:05.529202+0000 2024-04-25T17:00:13.662 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: created 2024-04-25T17:00:05.529202+0000 2024-04-25T17:00:13.663 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: min_mon_release 19 (squid) 2024-04-25T17:00:13.663 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: election_strategy: 1 2024-04-25T17:00:13.663 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T17:00:13.663 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: fsmap 2024-04-25T17:00:13.663 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T17:00:13.663 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:13 smithi096 ceph-mon[29131]: mgrmap e1: no daemons active 2024-04-25T17:00:14.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.776+0000 7f7b54534700 1 Processor -- start 2024-04-25T17:00:14.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.776+0000 7f7b54534700 1 -- start start 2024-04-25T17:00:14.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.777+0000 7f7b54534700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:14.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.777+0000 7f7b54534700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10a380 con 0x7f7b4c109980 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.777+0000 7f7b522d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.777+0000 7f7b522d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54830/0 (socket says 172.21.15.96:54830) 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.777+0000 7f7b522d0700 1 -- 172.21.15.96:0/2355582209 learned_addr learned my addr 172.21.15.96:0/2355582209 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.777+0000 7f7b522d0700 1 -- 172.21.15.96:0/2355582209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c10ab50 con 0x7f7b4c109980 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.778+0000 7f7b522d0700 1 --2- 172.21.15.96:0/2355582209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c109d90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b480098a0 tx=0x7f7b48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca61217a3449683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.778+0000 7f7b512ce700 1 -- 172.21.15.96:0/2355582209 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b48004160 con 0x7f7b4c109980 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.778+0000 7f7b512ce700 1 -- 172.21.15.96:0/2355582209 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b48004300 con 0x7f7b4c109980 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.778+0000 7f7b512ce700 1 -- 172.21.15.96:0/2355582209 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b48011af0 con 0x7f7b4c109980 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.778+0000 7f7b54534700 1 -- 172.21.15.96:0/2355582209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 msgr2=0x7f7b4c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.778+0000 7f7b54534700 1 --2- 172.21.15.96:0/2355582209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c109d90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b480098a0 tx=0x7f7b48009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 -- 172.21.15.96:0/2355582209 shutdown_connections 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 --2- 172.21.15.96:0/2355582209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c109d90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 -- 172.21.15.96:0/2355582209 >> 172.21.15.96:0/2355582209 conn(0x7f7b4c104fe0 msgr2=0x7f7b4c107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:14.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 -- 172.21.15.96:0/2355582209 shutdown_connections 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 -- 172.21.15.96:0/2355582209 wait complete. 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 Processor -- start 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 -- start start 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.779+0000 7f7b54534700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10df00 con 0x7f7b4c109980 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b522d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b522d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c19de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54844/0 (socket says 172.21.15.96:54844) 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b522d0700 1 -- 172.21.15.96:0/1554879350 learned_addr learned my addr 172.21.15.96:0/1554879350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b522d0700 1 -- 172.21.15.96:0/1554879350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c19e380 con 0x7f7b4c109980 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b522d0700 1 --2- 172.21.15.96:0/1554879350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c19de40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b48011670 tx=0x7f7b48004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b437fe700 1 -- 172.21.15.96:0/1554879350 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b480036a0 con 0x7f7b4c109980 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c19e610 con 0x7f7b4c109980 2024-04-25T17:00:14.403 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b437fe700 1 -- 172.21.15.96:0/1554879350 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b48003cd0 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b437fe700 1 -- 172.21.15.96:0/1554879350 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b480193c0 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1a1220 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b437fe700 1 -- 172.21.15.96:0/1554879350 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b48019a10 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.780+0000 7f7b437fe700 1 -- 172.21.15.96:0/1554879350 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b480243f0 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.781+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c064cf0 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.784+0000 7f7b437fe700 1 -- 172.21.15.96:0/1554879350 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7b48043e50 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.851+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7b4c1a13f0 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.853+0000 7f7b437fe700 1 -- 172.21.15.96:0/1554879350 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7b48032020 con 0x7f7b4c109980 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.854+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 msgr2=0x7f7b4c19de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.854+0000 7f7b54534700 1 --2- 172.21.15.96:0/1554879350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c19de40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b48011670 tx=0x7f7b48004280 comp rx=0 tx=0).stop 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.854+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 shutdown_connections 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.854+0000 7f7b54534700 1 --2- 172.21.15.96:0/1554879350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b4c109980 0x7f7b4c19de40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.854+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 >> 172.21.15.96:0/1554879350 conn(0x7f7b4c104fe0 msgr2=0x7f7b4c105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.854+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 shutdown_connections 2024-04-25T17:00:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:13.854+0000 7f7b54534700 1 -- 172.21.15.96:0/1554879350 wait complete. 2024-04-25T17:00:14.405 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T17:00:14.427 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:00:14.428 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mgr... 2024-04-25T17:00:14.428 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T17:00:14.428 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T17:00:14.674 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.x 2024-04-25T17:00:14.675 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T17:00:14.820 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1be09d02-0325-11ef-bc93-c7b262605968.target.wants/ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1be09d02-0325-11ef-bc93-c7b262605968@.service. 2024-04-25T17:00:14.837 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:14 smithi096 systemd[1]: Starting Ceph mgr.x for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:00:15.176 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:14 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1554879350' entity='client.admin' 2024-04-25T17:00:15.176 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 podman[29346]: 2024-04-25 17:00:14.969818241 +0000 UTC m=+0.017556490 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T17:00:15.176 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 podman[29346]: 2024-04-25 17:00:15.074754803 +0000 UTC m=+0.122493088 container create a92cedda03dffebef043e87f9f232360ef1e6122b009f3621c7e11bba0f4223e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:00:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T17:00:15.356 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T17:00:15.364 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T17:00:15.364 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T17:00:15.364 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T17:00:15.364 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T17:00:15.372 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T17:00:15.372 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T17:00:15.379 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T17:00:15.379 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T17:00:15.380 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T17:00:15.380 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T17:00:15.380 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr to start... 2024-04-25T17:00:15.380 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr... 2024-04-25T17:00:15.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 podman[29346]: 2024-04-25 17:00:15.285354792 +0000 UTC m=+0.333093035 container init a92cedda03dffebef043e87f9f232360ef1e6122b009f3621c7e11bba0f4223e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T17:00:15.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 podman[29346]: 2024-04-25 17:00:15.28815636 +0000 UTC m=+0.335894605 container start a92cedda03dffebef043e87f9f232360ef1e6122b009f3621c7e11bba0f4223e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T17:00:15.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 bash[29346]: a92cedda03dffebef043e87f9f232360ef1e6122b009f3621c7e11bba0f4223e 2024-04-25T17:00:15.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 systemd[1]: Started Ceph mgr.x for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:00:15.798 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:15.453+0000 7f743474c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:00:15.798 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:15.642+0000 7f743474c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:00:15.798 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:15.747+0000 7f743474c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:00:16.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:16 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:16.053+0000 7f743474c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:00:16.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:16 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:16.138+0000 7f743474c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:00:17.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:16 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3123715146' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T17:00:17.685 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:17.348+0000 7f743474c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:00:17.685 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:17.518+0000 7f743474c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:00:17.685 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:17.688+0000 7f743474c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:00:18.262 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:17.976+0000 7f743474c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:00:18.262 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:18.066+0000 7f743474c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:00:18.262 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:18.264+0000 7f743474c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:00:18.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:18.453+0000 7f743474c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:00:19.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:19.048+0000 7f743474c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:00:19.761 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:19.467+0000 7f743474c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:00:19.761 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:19.561+0000 7f743474c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:00:19.761 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:19.764+0000 7f743474c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:00:20.625 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:20.279+0000 7f743474c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:00:20.625 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:20.370+0000 7f743474c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:00:20.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:20.608+0000 7f743474c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:00:20.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:20.740+0000 7f743474c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:00:20.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:20.834+0000 7f743474c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:00:21.243 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:20.948+0000 7f743474c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:00:21.243 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:21.245+0000 7f743474c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:00:22.111 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:21.840+0000 7f743474c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:00:22.111 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:22.058+0000 7f743474c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:00:22.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: Activating manager daemon x 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: mgrmap e2: x(active, starting, since 0.00363111s) 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: Manager daemon x is now available 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' 2024-04-25T17:00:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:22 smithi096 ceph-mon[29131]: from='mgr.14100 172.21.15.96:0/2600719317' entity='mgr.x' 2024-04-25T17:00:23.380 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "1be09d02-0325-11ef-bc93-c7b262605968", 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T17:00:23.384 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T17:00:23.385 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T17:00:23.386 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:23.386 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T17:00:23.386 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:23.386 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T17:00:23.386 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T17:00:23.386 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T17:00:23.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T17:00:23.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T17:00:23.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T17:00:23.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:23.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T17:00:23.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T17:00:23.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T17:00:08.429845+0000", 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.629+0000 7f33f8b6e700 1 Processor -- start 2024-04-25T17:00:23.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.629+0000 7f33f8b6e700 1 -- start start 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.630+0000 7f33f8b6e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f410c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.630+0000 7f33f8b6e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33f407d650 con 0x7f33f410a540 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.630+0000 7f33f259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f410c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.630+0000 7f33f259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f410c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54860/0 (socket says 172.21.15.96:54860) 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.630+0000 7f33f259c700 1 -- 172.21.15.96:0/3649108264 learned_addr learned my addr 172.21.15.96:0/3649108264 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.630+0000 7f33f259c700 1 -- 172.21.15.96:0/3649108264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f410cea0 con 0x7f33f410a540 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.630+0000 7f33f259c700 1 --2- 172.21.15.96:0/3649108264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f410c960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f33e40098a0 tx=0x7f33e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e738613eb1edc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.631+0000 7f33f159a700 1 -- 172.21.15.96:0/3649108264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33e4004160 con 0x7f33f410a540 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.631+0000 7f33f159a700 1 -- 172.21.15.96:0/3649108264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f33e4004300 con 0x7f33f410a540 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.631+0000 7f33f159a700 1 -- 172.21.15.96:0/3649108264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33e4011af0 con 0x7f33f410a540 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.632+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3649108264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 msgr2=0x7f33f410c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.632+0000 7f33f8b6e700 1 --2- 172.21.15.96:0/3649108264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f410c960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f33e40098a0 tx=0x7f33e4009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.632+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3649108264 shutdown_connections 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.632+0000 7f33f8b6e700 1 --2- 172.21.15.96:0/3649108264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f410c960 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:23.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.632+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3649108264 >> 172.21.15.96:0/3649108264 conn(0x7f33f4103f40 msgr2=0x7f33f4106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.632+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3649108264 shutdown_connections 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.632+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3649108264 wait complete. 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f8b6e700 1 Processor -- start 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f8b6e700 1 -- start start 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f8b6e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f8b6e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33f410dda0 con 0x7f33f410a540 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54870/0 (socket says 172.21.15.96:54870) 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f259c700 1 -- 172.21.15.96:0/3123715146 learned_addr learned my addr 172.21.15.96:0/3123715146 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.633+0000 7f33f259c700 1 -- 172.21.15.96:0/3123715146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f419e340 con 0x7f33f410a540 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33f259c700 1 --2- 172.21.15.96:0/3123715146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f419de00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f33e40117b0 tx=0x7f33e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33e37fe700 1 -- 172.21.15.96:0/3123715146 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33e40039c0 con 0x7f33f410a540 2024-04-25T17:00:23.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33e37fe700 1 -- 172.21.15.96:0/3123715146 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f33e4003b60 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f419e5d0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33e37fe700 1 -- 172.21.15.96:0/3123715146 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33e40196a0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f41a11e0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33e37fe700 1 -- 172.21.15.96:0/3123715146 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f33e4019cf0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.634+0000 7f33e37fe700 1 -- 172.21.15.96:0/3123715146 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f33e4024730 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.635+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f4064cf0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.637+0000 7f33e37fe700 1 -- 172.21.15.96:0/3123715146 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f33e40042a0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.710+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f33f41a13b0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.710+0000 7f33e37fe700 1 -- 172.21.15.96:0/3123715146 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f33e401abb0 con 0x7f33f410a540 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.711+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 msgr2=0x7f33f419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.711+0000 7f33f8b6e700 1 --2- 172.21.15.96:0/3123715146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f419de00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f33e40117b0 tx=0x7f33e4011b70 comp rx=0 tx=0).stop 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.711+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 shutdown_connections 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.711+0000 7f33f8b6e700 1 --2- 172.21.15.96:0/3123715146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33f410a540 0x7f33f419de00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.711+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 >> 172.21.15.96:0/3123715146 conn(0x7f33f4103f40 msgr2=0x7f33f4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:23.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.711+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 shutdown_connections 2024-04-25T17:00:23.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:16.711+0000 7f33f8b6e700 1 -- 172.21.15.96:0/3123715146 wait complete. 2024-04-25T17:00:23.400 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (1/15)... 2024-04-25T17:00:24.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:24 smithi096 ceph-mon[29131]: mgrmap e3: x(active, since 1.00833s) 2024-04-25T17:00:25.337 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:25 smithi096 ceph-mon[29131]: mgrmap e4: x(active, since 2s) 2024-04-25T17:00:26.562 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:26 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/4111046674' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T17:00:26.754 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:26.754 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "1be09d02-0325-11ef-bc93-c7b262605968", 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T17:00:26.755 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T17:00:26.756 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T17:00:26.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T17:00:26.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T17:00:26.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:26.765 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:26.766 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T17:00:08.429845+0000", 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.957+0000 7f9444f7d700 1 Processor -- start 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.957+0000 7f9444f7d700 1 -- start start 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.957+0000 7f9444f7d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944010c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.957+0000 7f9444f7d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f944007d870 con 0x7f944010a460 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.957+0000 7f943e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944010c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.957+0000 7f943e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944010c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36868/0 (socket says 172.21.15.96:36868) 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.957+0000 7f943e59c700 1 -- 172.21.15.96:0/3844193738 learned_addr learned my addr 172.21.15.96:0/3844193738 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:26.767 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.958+0000 7f943e59c700 1 -- 172.21.15.96:0/3844193738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944010cdc0 con 0x7f944010a460 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.958+0000 7f943e59c700 1 --2- 172.21.15.96:0/3844193738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944010c880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94300098a0 tx=0x7f9430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c9a108d7b39676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.958+0000 7f943d59a700 1 -- 172.21.15.96:0/3844193738 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9430004160 con 0x7f944010a460 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.958+0000 7f943d59a700 1 -- 172.21.15.96:0/3844193738 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9430004300 con 0x7f944010a460 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.958+0000 7f943d59a700 1 -- 172.21.15.96:0/3844193738 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9430011af0 con 0x7f944010a460 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.959+0000 7f9444f7d700 1 -- 172.21.15.96:0/3844193738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 msgr2=0x7f944010c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.959+0000 7f9444f7d700 1 --2- 172.21.15.96:0/3844193738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944010c880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94300098a0 tx=0x7f9430009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.959+0000 7f9444f7d700 1 -- 172.21.15.96:0/3844193738 shutdown_connections 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.959+0000 7f9444f7d700 1 --2- 172.21.15.96:0/3844193738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944010c880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.959+0000 7f9444f7d700 1 -- 172.21.15.96:0/3844193738 >> 172.21.15.96:0/3844193738 conn(0x7f9440103e80 msgr2=0x7f9440106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.959+0000 7f9444f7d700 1 -- 172.21.15.96:0/3844193738 shutdown_connections 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.959+0000 7f9444f7d700 1 -- 172.21.15.96:0/3844193738 wait complete. 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f9444f7d700 1 Processor -- start 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f9444f7d700 1 -- start start 2024-04-25T17:00:26.768 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f9444f7d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f9444f7d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f944010dd20 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f943e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f943e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944019dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36880/0 (socket says 172.21.15.96:36880) 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f943e59c700 1 -- 172.21.15.96:0/4111046674 learned_addr learned my addr 172.21.15.96:0/4111046674 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f943e59c700 1 -- 172.21.15.96:0/4111046674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944019e2d0 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.960+0000 7f943e59c700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944019dd90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9430009fb0 tx=0x7f9430011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.961+0000 7f942f7fe700 1 -- 172.21.15.96:0/4111046674 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9430011bc0 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.961+0000 7f942f7fe700 1 -- 172.21.15.96:0/4111046674 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9430024440 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.961+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944019e560 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.961+0000 7f942f7fe700 1 -- 172.21.15.96:0/4111046674 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f943001a5c0 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.961+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94401a1170 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.962+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9440064cf0 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.962+0000 7f942f7fe700 1 -- 172.21.15.96:0/4111046674 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f943001a880 con 0x7f944010a460 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.962+0000 7f942f7fe700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f942803d4d0 0x7f942803f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:26.769 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.962+0000 7f942f7fe700 1 -- 172.21.15.96:0/4111046674 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9430052590 con 0x7f944010a460 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.962+0000 7f943dd9b700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f942803d4d0 0x7f942803f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.963+0000 7f943dd9b700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f942803d4d0 0x7f942803f9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9434006fd0 tx=0x7f9434006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:25.967+0000 7f942f7fe700 1 -- 172.21.15.96:0/4111046674 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f943000b3e0 con 0x7f944010a460 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.255+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f94401a1460 con 0x7f944010a460 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.255+0000 7f942f7fe700 1 -- 172.21.15.96:0/4111046674 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9430029df0 con 0x7f944010a460 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f942803d4d0 msgr2=0x7f942803f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f942803d4d0 0x7f942803f9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9434006fd0 tx=0x7f9434006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 msgr2=0x7f944019dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944019dd90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9430009fb0 tx=0x7f9430011890 comp rx=0 tx=0).stop 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 shutdown_connections 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f942803d4d0 0x7f942803f9e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 --2- 172.21.15.96:0/4111046674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944010a460 0x7f944019dd90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:26.770 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 >> 172.21.15.96:0/4111046674 conn(0x7f9440103e80 msgr2=0x7f9440104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:26.771 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 shutdown_connections 2024-04-25T17:00:26.771 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:26.258+0000 7f9444f7d700 1 -- 172.21.15.96:0/4111046674 wait complete. 2024-04-25T17:00:26.771 INFO:teuthology.orchestra.run.smithi096.stdout:mgr is available 2024-04-25T17:00:27.869 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:27 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2762529666' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T17:00:27.925 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:27.932 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T17:00:27.932 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:27.932 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T17:00:27.932 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.196+0000 7fb777ee2700 1 Processor -- start 2024-04-25T17:00:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.196+0000 7fb777ee2700 1 -- start start 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.196+0000 7fb777ee2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77010c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.196+0000 7fb777ee2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb77007d6d0 con 0x7fb77010a490 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.196+0000 7fb775c7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77010c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.196+0000 7fb775c7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77010c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36890/0 (socket says 172.21.15.96:36890) 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.196+0000 7fb775c7e700 1 -- 172.21.15.96:0/4120763535 learned_addr learned my addr 172.21.15.96:0/4120763535 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.197+0000 7fb775c7e700 1 -- 172.21.15.96:0/4120763535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77010cdf0 con 0x7fb77010a490 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.197+0000 7fb775c7e700 1 --2- 172.21.15.96:0/4120763535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77010c8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0098a0 tx=0x7fb76c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e96d36265c5d92f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.197+0000 7fb774c7c700 1 -- 172.21.15.96:0/4120763535 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c004160 con 0x7fb77010a490 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.197+0000 7fb774c7c700 1 -- 172.21.15.96:0/4120763535 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb76c004300 con 0x7fb77010a490 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.197+0000 7fb774c7c700 1 -- 172.21.15.96:0/4120763535 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c011af0 con 0x7fb77010a490 2024-04-25T17:00:27.934 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.198+0000 7fb777ee2700 1 -- 172.21.15.96:0/4120763535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 msgr2=0x7fb77010c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.198+0000 7fb777ee2700 1 --2- 172.21.15.96:0/4120763535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77010c8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0098a0 tx=0x7fb76c009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.198+0000 7fb777ee2700 1 -- 172.21.15.96:0/4120763535 shutdown_connections 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.198+0000 7fb777ee2700 1 --2- 172.21.15.96:0/4120763535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77010c8b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.198+0000 7fb777ee2700 1 -- 172.21.15.96:0/4120763535 >> 172.21.15.96:0/4120763535 conn(0x7fb770103ed0 msgr2=0x7fb770106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.198+0000 7fb777ee2700 1 -- 172.21.15.96:0/4120763535 shutdown_connections 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.198+0000 7fb777ee2700 1 -- 172.21.15.96:0/4120763535 wait complete. 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb777ee2700 1 Processor -- start 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb777ee2700 1 -- start start 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb777ee2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb777ee2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010dd50 con 0x7fb77010a490 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb775c7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:27.935 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb775c7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36900/0 (socket says 172.21.15.96:36900) 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb775c7e700 1 -- 172.21.15.96:0/2762529666 learned_addr learned my addr 172.21.15.96:0/2762529666 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb775c7e700 1 -- 172.21.15.96:0/2762529666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77019e2e0 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb775c7e700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77019dda0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0117b0 tx=0x7fb76c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb766ffd700 1 -- 172.21.15.96:0/2762529666 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c0039c0 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.199+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77019e570 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.200+0000 7fb766ffd700 1 -- 172.21.15.96:0/2762529666 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb76c003b60 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.200+0000 7fb766ffd700 1 -- 172.21.15.96:0/2762529666 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c0196a0 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.200+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a1180 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.200+0000 7fb766ffd700 1 -- 172.21.15.96:0/2762529666 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7fb76c019960 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.201+0000 7fb766ffd700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fb75c03d520 0x7fb75c03fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.201+0000 7fb766ffd700 1 -- 172.21.15.96:0/2762529666 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb76c053930 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.201+0000 7fb77547d700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fb75c03d520 0x7fb75c03fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.201+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb770064cf0 con 0x7fb77010a490 2024-04-25T17:00:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.201+0000 7fb77547d700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fb75c03d520 0x7fb75c03fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb760006fd0 tx=0x7fb760006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.206+0000 7fb766ffd700 1 -- 172.21.15.96:0/2762529666 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb76c00b3e0 con 0x7fb77010a490 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.400+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb7701a1350 con 0x7fb77010a490 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.402+0000 7fb766ffd700 1 -- 172.21.15.96:0/2762529666 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fb76c029260 con 0x7fb77010a490 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fb75c03d520 msgr2=0x7fb75c03fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fb75c03d520 0x7fb75c03fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb760006fd0 tx=0x7fb760006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 msgr2=0x7fb77019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77019dda0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0117b0 tx=0x7fb76c011b70 comp rx=0 tx=0).stop 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 shutdown_connections 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fb75c03d520 0x7fb75c03fa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 --2- 172.21.15.96:0/2762529666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77010a490 0x7fb77019dda0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 >> 172.21.15.96:0/2762529666 conn(0x7fb770103ed0 msgr2=0x7fb770104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 shutdown_connections 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:27.404+0000 7fb777ee2700 1 -- 172.21.15.96:0/2762529666 wait complete. 2024-04-25T17:00:27.937 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling cephadm module... 2024-04-25T17:00:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:28 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2325062095' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T17:00:28.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:28 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: ignoring --setuser ceph since I am not root 2024-04-25T17:00:28.937 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:28 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: ignoring --setgroup ceph since I am not root 2024-04-25T17:00:28.937 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:28 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:28.894+0000 7fa0533ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:00:29.281 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:29 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:29.081+0000 7fa0533ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:00:29.282 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:29 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:29.183+0000 7fa0533ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:00:29.287 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.388+0000 7fe3819d4700 1 Processor -- start 2024-04-25T17:00:29.287 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.388+0000 7fe3819d4700 1 -- start start 2024-04-25T17:00:29.287 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.388+0000 7fe3819d4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c109da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:29.287 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.388+0000 7fe3819d4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10a390 con 0x7fe37c109990 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe37affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe37affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c109da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49702/0 (socket says 172.21.15.96:49702) 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe37affd700 1 -- 172.21.15.96:0/3037422422 learned_addr learned my addr 172.21.15.96:0/3037422422 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe37affd700 1 -- 172.21.15.96:0/3037422422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c10ab60 con 0x7fe37c109990 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe37affd700 1 --2- 172.21.15.96:0/3037422422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c109da0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009e80 tx=0x7fe36c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60c118f2c7cfe71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe379ffb700 1 -- 172.21.15.96:0/3037422422 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe36c017070 con 0x7fe37c109990 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe379ffb700 1 -- 172.21.15.96:0/3037422422 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe36c00b8c0 con 0x7fe37c109990 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.389+0000 7fe379ffb700 1 -- 172.21.15.96:0/3037422422 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe36c004490 con 0x7fe37c109990 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.390+0000 7fe3819d4700 1 -- 172.21.15.96:0/3037422422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 msgr2=0x7fe37c109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.390+0000 7fe3819d4700 1 --2- 172.21.15.96:0/3037422422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c109da0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009e80 tx=0x7fe36c00b330 comp rx=0 tx=0).stop 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.390+0000 7fe3819d4700 1 -- 172.21.15.96:0/3037422422 shutdown_connections 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.390+0000 7fe3819d4700 1 --2- 172.21.15.96:0/3037422422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c109da0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.390+0000 7fe3819d4700 1 -- 172.21.15.96:0/3037422422 >> 172.21.15.96:0/3037422422 conn(0x7fe37c104ff0 msgr2=0x7fe37c107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:29.288 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.390+0000 7fe3819d4700 1 -- 172.21.15.96:0/3037422422 shutdown_connections 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.390+0000 7fe3819d4700 1 -- 172.21.15.96:0/3037422422 wait complete. 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.391+0000 7fe3819d4700 1 Processor -- start 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.391+0000 7fe3819d4700 1 -- start start 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.391+0000 7fe3819d4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.391+0000 7fe3819d4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10df10 con 0x7fe37c109990 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.391+0000 7fe37affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.391+0000 7fe37affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c19ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49714/0 (socket says 172.21.15.96:49714) 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.391+0000 7fe37affd700 1 -- 172.21.15.96:0/2325062095 learned_addr learned my addr 172.21.15.96:0/2325062095 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.392+0000 7fe37affd700 1 -- 172.21.15.96:0/2325062095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c19e330 con 0x7fe37c109990 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.392+0000 7fe37affd700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c19ddf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009fd0 tx=0x7fe36c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.392+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe36c020020 con 0x7fe37c109990 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.392+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe36c012430 con 0x7fe37c109990 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.392+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c19e5c0 con 0x7fe37c109990 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.392+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe36c017040 con 0x7fe37c109990 2024-04-25T17:00:29.289 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.392+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a11d0 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.393+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7fe36c023020 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.393+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c064cf0 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.393+0000 7fe3809d2700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fe36403d520 0x7fe36403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.393+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe36c00f050 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.393+0000 7fe37a7fc700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fe36403d520 0x7fe36403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.393+0000 7fe37a7fc700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fe36403d520 0x7fe36403fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe370006fd0 tx=0x7fe370006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.398+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe36c02a020 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.650+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe37c1a1850 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.702+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe36c02b260 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.703+0000 7fe3809d2700 1 -- 172.21.15.96:0/2325062095 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fe36c0243b0 con 0x7fe37c109990 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.704+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fe36403d520 msgr2=0x7fe36403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.704+0000 7fe3819d4700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fe36403d520 0x7fe36403fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe370006fd0 tx=0x7fe370006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.704+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 msgr2=0x7fe37c19ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:29.290 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.704+0000 7fe3819d4700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c19ddf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009fd0 tx=0x7fe36c00bd50 comp rx=0 tx=0).stop 2024-04-25T17:00:29.291 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.704+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 shutdown_connections 2024-04-25T17:00:29.291 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.705+0000 7fe3819d4700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fe36403d520 0x7fe36403fa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:29.291 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.705+0000 7fe3819d4700 1 --2- 172.21.15.96:0/2325062095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe37c109990 0x7fe37c19ddf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:29.291 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.705+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 >> 172.21.15.96:0/2325062095 conn(0x7fe37c104ff0 msgr2=0x7fe37c1058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:29.291 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.705+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 shutdown_connections 2024-04-25T17:00:29.291 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:28.705+0000 7fe3819d4700 1 -- 172.21.15.96:0/2325062095 wait complete. 2024-04-25T17:00:29.580 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:29 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:29.496+0000 7fa0533ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:00:29.580 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:29 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:29.581+0000 7fa0533ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:00:29.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:29 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2325062095' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T17:00:29.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:29 smithi096 ceph-mon[29131]: mgrmap e5: x(active, since 6s) 2024-04-25T17:00:30.956 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:30 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/4239676419' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T17:00:30.956 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:30 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:30.770+0000 7fa0533ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:00:30.957 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:30 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:30.958+0000 7fa0533ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:00:31.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:31 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:31.128+0000 7fa0533ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:00:31.910 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:31 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:31.418+0000 7fa0533ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:00:31.910 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:31 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:31.510+0000 7fa0533ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:00:31.910 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:31 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:31.715+0000 7fa0533ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:00:32.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:31 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:31.912+0000 7fa0533ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:00:32.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T17:00:32.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T17:00:32.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T17:00:32.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T17:00:32.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T17:00:32.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.487+0000 7f4e46196700 1 Processor -- start 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.487+0000 7f4e46196700 1 -- start start 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e46196700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e40108040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e46196700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40108610 con 0x7f4e40107c30 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e40108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e40108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49730/0 (socket says 172.21.15.96:49730) 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3f7fe700 1 -- 172.21.15.96:0/1868976081 learned_addr learned my addr 172.21.15.96:0/1868976081 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3f7fe700 1 -- 172.21.15.96:0/1868976081 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4010ac30 con 0x7f4e40107c30 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3f7fe700 1 --2- 172.21.15.96:0/1868976081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e40108040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4e340098a0 tx=0x7f4e34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfea39199517c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3e7fc700 1 -- 172.21.15.96:0/1868976081 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e34004160 con 0x7f4e40107c30 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3e7fc700 1 -- 172.21.15.96:0/1868976081 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e34004300 con 0x7f4e40107c30 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.488+0000 7f4e3e7fc700 1 -- 172.21.15.96:0/1868976081 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e34011af0 con 0x7f4e40107c30 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.489+0000 7f4e46196700 1 -- 172.21.15.96:0/1868976081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 msgr2=0x7f4e40108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:32.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.489+0000 7f4e46196700 1 --2- 172.21.15.96:0/1868976081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e40108040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4e340098a0 tx=0x7f4e34009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.490+0000 7f4e46196700 1 -- 172.21.15.96:0/1868976081 shutdown_connections 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.490+0000 7f4e46196700 1 --2- 172.21.15.96:0/1868976081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e40108040 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.490+0000 7f4e46196700 1 -- 172.21.15.96:0/1868976081 >> 172.21.15.96:0/1868976081 conn(0x7f4e40103660 msgr2=0x7f4e40105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.490+0000 7f4e46196700 1 -- 172.21.15.96:0/1868976081 shutdown_connections 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.490+0000 7f4e46196700 1 -- 172.21.15.96:0/1868976081 wait complete. 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.490+0000 7f4e46196700 1 Processor -- start 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e46196700 1 -- start start 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e46196700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e4019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e46196700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4010de60 con 0x7f4e40107c30 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e3f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e4019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e3f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e4019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49746/0 (socket says 172.21.15.96:49746) 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e3f7fe700 1 -- 172.21.15.96:0/4239676419 learned_addr learned my addr 172.21.15.96:0/4239676419 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e3f7fe700 1 -- 172.21.15.96:0/4239676419 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4019e560 con 0x7f4e40107c30 2024-04-25T17:00:32.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.491+0000 7f4e3f7fe700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e4019e020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e34011670 tx=0x7f4e34004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.492+0000 7f4e3cff9700 1 -- 172.21.15.96:0/4239676419 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e340036a0 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.492+0000 7f4e3cff9700 1 -- 172.21.15.96:0/4239676419 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e3401a430 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.492+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4019e7f0 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.492+0000 7f4e3cff9700 1 -- 172.21.15.96:0/4239676419 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e34019550 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.492+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e401a1400 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.493+0000 7f4e3cff9700 1 -- 172.21.15.96:0/4239676419 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f4e34019810 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.493+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e40064cf0 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.493+0000 7f4e3cff9700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 0x7f4e2803fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.493+0000 7f4e3cff9700 1 -- 172.21.15.96:0/4239676419 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e34052900 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.493+0000 7f4e3effd700 1 -- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 msgr2=0x7f4e2803fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/4227184783 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.493+0000 7f4e3effd700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 0x7f4e2803fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.498+0000 7f4e3cff9700 1 -- 172.21.15.96:0/4239676419 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e34017e90 con 0x7f4e40107c30 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.694+0000 7f4e3effd700 1 -- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 msgr2=0x7f4e2803fa30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/4227184783 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.694+0000 7f4e3effd700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 0x7f4e2803fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T17:00:32.641 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.756+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4e4002e690 con 0x7f4e40107c30 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.756+0000 7f4e3cff9700 1 -- 172.21.15.96:0/4239676419 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f4e34023390 con 0x7f4e40107c30 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 msgr2=0x7f4e2803fa30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 0x7f4e2803fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 msgr2=0x7f4e4019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e4019e020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e34011670 tx=0x7f4e34004280 comp rx=0 tx=0).stop 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 shutdown_connections 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7f4e2803d520 0x7f4e2803fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 --2- 172.21.15.96:0/4239676419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e40107c30 0x7f4e4019e020 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 >> 172.21.15.96:0/4239676419 conn(0x7f4e40103660 msgr2=0x7f4e40104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 shutdown_connections 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:30.759+0000 7f4e46196700 1 -- 172.21.15.96:0/4239676419 wait complete. 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-04-25T17:00:32.642 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 5... 2024-04-25T17:00:32.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:32 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:32.514+0000 7fa0533ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:00:33.034 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:32 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:32.950+0000 7fa0533ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:00:33.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:33 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:33.053+0000 7fa0533ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:00:33.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:33 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:33.256+0000 7fa0533ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:00:33.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:33 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:33.774+0000 7fa0533ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:00:33.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:33 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:33.867+0000 7fa0533ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:00:34.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:34.111+0000 7fa0533ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:00:34.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:34.246+0000 7fa0533ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:00:34.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:34.340+0000 7fa0533ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:00:34.754 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:34.455+0000 7fa0533ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:00:34.754 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:34.756+0000 7fa0533ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:00:35.622 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:35.354+0000 7fa0533ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:00:35.622 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:35.571+0000 7fa0533ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:00:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: Active manager daemon x restarted 2024-04-25T17:00:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: Activating manager daemon x 2024-04-25T17:00:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: mgrmap e6: x(active, starting, since 0.00394495s) 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: Manager daemon x is now available 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:00:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:35 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:00:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:36 smithi096 ceph-mon[29131]: Found migration_current of "None". Setting to last migration. 2024-04-25T17:00:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:36 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T17:00:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:36 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T17:00:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:36 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:36.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:36 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:36.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:36 smithi096 ceph-mon[29131]: mgrmap e7: x(active, since 1.00696s) 2024-04-25T17:00:37.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T17:00:37.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T17:00:37.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T17:00:37.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.045+0000 7fd8360dd700 1 Processor -- start 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.045+0000 7fd8360dd700 1 -- start start 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd8360dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd8360dd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd83007ce00 con 0x7fd830108290 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49762/0 (socket says 172.21.15.96:49762) 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd82f7fe700 1 -- 172.21.15.96:0/984687156 learned_addr learned my addr 172.21.15.96:0/984687156 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd82f7fe700 1 -- 172.21.15.96:0/984687156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd830108be0 con 0x7fd830108290 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd82f7fe700 1 --2- 172.21.15.96:0/984687156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301086a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91540c51849ba4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.046+0000 7fd82e7fc700 1 -- 172.21.15.96:0/984687156 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820004160 con 0x7fd830108290 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd82e7fc700 1 -- 172.21.15.96:0/984687156 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd820004300 con 0x7fd830108290 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd82e7fc700 1 -- 172.21.15.96:0/984687156 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820011af0 con 0x7fd830108290 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd8360dd700 1 -- 172.21.15.96:0/984687156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 msgr2=0x7fd8301086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:37.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd8360dd700 1 --2- 172.21.15.96:0/984687156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301086a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd820009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd8360dd700 1 -- 172.21.15.96:0/984687156 shutdown_connections 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd8360dd700 1 --2- 172.21.15.96:0/984687156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301086a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd8360dd700 1 -- 172.21.15.96:0/984687156 >> 172.21.15.96:0/984687156 conn(0x7fd830103f00 msgr2=0x7fd830106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.047+0000 7fd8360dd700 1 -- 172.21.15.96:0/984687156 shutdown_connections 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.048+0000 7fd8360dd700 1 -- 172.21.15.96:0/984687156 wait complete. 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.048+0000 7fd8360dd700 1 Processor -- start 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.048+0000 7fd8360dd700 1 -- start start 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd8360dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd8360dd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd830109c60 con 0x7fd830108290 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49766/0 (socket says 172.21.15.96:49766) 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82f7fe700 1 -- 172.21.15.96:0/2560265309 learned_addr learned my addr 172.21.15.96:0/2560265309 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82f7fe700 1 -- 172.21.15.96:0/2560265309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301a05c0 con 0x7fd830108290 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82f7fe700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301a0080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd820009fb0 tx=0x7fd820011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820011bc0 con 0x7fd830108290 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd820024440 con 0x7fd830108290 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.049+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd82001a5c0 con 0x7fd830108290 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.050+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301a0850 con 0x7fd830108290 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.050+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a1290 con 0x7fd830108290 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.051+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fd82001a830 con 0x7fd830108290 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.051+0000 7fd82cff9700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 0x7fd81803fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.051+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 --> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd818040190 con 0x7fd81803d570 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.051+0000 7fd82effd700 1 -- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 msgr2=0x7fd81803fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/4227184783 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.051+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd8200525e0 con 0x7fd830108290 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.051+0000 7fd82effd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 0x7fd81803fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.252+0000 7fd82effd700 1 -- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 msgr2=0x7fd81803fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/4227184783 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.252+0000 7fd82effd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 0x7fd81803fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.653+0000 7fd82effd700 1 -- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 msgr2=0x7fd81803fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/4227184783 2024-04-25T17:00:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:33.653+0000 7fd82effd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 0x7fd81803fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:34.454+0000 7fd82effd700 1 -- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 msgr2=0x7fd81803fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/4227184783 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:34.454+0000 7fd82effd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 0x7fd81803fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:35.575+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7fd8200510a0 con 0x7fd830108290 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:35.575+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 msgr2=0x7fd81803fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:35.575+0000 7fd82cff9700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 0x7fd81803fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.577+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7fd8200245f0 con 0x7fd830108290 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.577+0000 7fd82cff9700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fd818040eb0 0x7fd8180432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.577+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd820021ec0 con 0x7fd818040eb0 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.577+0000 7fd82effd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fd818040eb0 0x7fd8180432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.578+0000 7fd82effd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fd818040eb0 0x7fd8180432d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd824003a10 tx=0x7fd8240092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.579+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd820021ec0 con 0x7fd818040eb0 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.587+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd830064cf0 con 0x7fd818040eb0 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.587+0000 7fd82cff9700 1 -- 172.21.15.96:0/2560265309 <== mgr.14116 v2:172.21.15.96:6800/1771523969 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd830064cf0 con 0x7fd818040eb0 2024-04-25T17:00:37.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fd818040eb0 msgr2=0x7fd8180432d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fd818040eb0 0x7fd8180432d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd824003a10 tx=0x7fd8240092b0 comp rx=0 tx=0).stop 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 msgr2=0x7fd8301a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301a0080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd820009fb0 tx=0x7fd820011890 comp rx=0 tx=0).stop 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 shutdown_connections 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fd818040eb0 0x7fd8180432d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd830108290 0x7fd8301a0080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 --2- 172.21.15.96:0/2560265309 >> [v2:172.21.15.96:6800/4227184783,v1:172.21.15.96:6801/4227184783] conn(0x7fd81803d570 0x7fd81803fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 >> 172.21.15.96:0/2560265309 conn(0x7fd830103f00 msgr2=0x7fd830193f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 shutdown_connections 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:36.588+0000 7fd8360dd700 1 -- 172.21.15.96:0/2560265309 wait complete. 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 5 is available 2024-04-25T17:00:37.484 INFO:teuthology.orchestra.run.smithi096.stdout:Setting orchestrator backend to cephadm... 2024-04-25T17:00:37.864 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:37 smithi096 ceph-mon[29131]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T17:00:37.864 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:37 smithi096 ceph-mon[29131]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T17:00:37.864 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:37 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:00:38.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.904+0000 7fc88cd5b700 1 Processor -- start 2024-04-25T17:00:38.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.904+0000 7fc88cd5b700 1 -- start start 2024-04-25T17:00:38.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.904+0000 7fc88cd5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc8881080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.904+0000 7fc88cd5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc888108670 con 0x7fc888107c90 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.904+0000 7fc88659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc8881080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.904+0000 7fc88659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc8881080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42498/0 (socket says 172.21.15.96:42498) 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.904+0000 7fc88659c700 1 -- 172.21.15.96:0/1281920572 learned_addr learned my addr 172.21.15.96:0/1281920572 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.905+0000 7fc88659c700 1 -- 172.21.15.96:0/1281920572 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88810ac90 con 0x7fc888107c90 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.905+0000 7fc88659c700 1 --2- 172.21.15.96:0/1281920572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc8881080a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc8780098a0 tx=0x7fc878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64d1ba0e770cbd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.905+0000 7fc88559a700 1 -- 172.21.15.96:0/1281920572 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc878004160 con 0x7fc888107c90 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.905+0000 7fc88559a700 1 -- 172.21.15.96:0/1281920572 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc878004300 con 0x7fc888107c90 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.905+0000 7fc88559a700 1 -- 172.21.15.96:0/1281920572 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc878011af0 con 0x7fc888107c90 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.906+0000 7fc88cd5b700 1 -- 172.21.15.96:0/1281920572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 msgr2=0x7fc8881080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.906+0000 7fc88cd5b700 1 --2- 172.21.15.96:0/1281920572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc8881080a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc8780098a0 tx=0x7fc878009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:38.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.906+0000 7fc88cd5b700 1 -- 172.21.15.96:0/1281920572 shutdown_connections 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.906+0000 7fc88cd5b700 1 --2- 172.21.15.96:0/1281920572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc8881080a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.906+0000 7fc88cd5b700 1 -- 172.21.15.96:0/1281920572 >> 172.21.15.96:0/1281920572 conn(0x7fc8881036a0 msgr2=0x7fc888105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.906+0000 7fc88cd5b700 1 -- 172.21.15.96:0/1281920572 shutdown_connections 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.906+0000 7fc88cd5b700 1 -- 172.21.15.96:0/1281920572 wait complete. 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88cd5b700 1 Processor -- start 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88cd5b700 1 -- start start 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88cd5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc88819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88cd5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810dec0 con 0x7fc888107c90 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc88819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc88819e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42514/0 (socket says 172.21.15.96:42514) 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88659c700 1 -- 172.21.15.96:0/2220612761 learned_addr learned my addr 172.21.15.96:0/2220612761 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88659c700 1 -- 172.21.15.96:0/2220612761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88819e580 con 0x7fc888107c90 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.907+0000 7fc88659c700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc88819e040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc878011670 tx=0x7fc878004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:38.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.908+0000 7fc8777fe700 1 -- 172.21.15.96:0/2220612761 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8780036a0 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.908+0000 7fc8777fe700 1 -- 172.21.15.96:0/2220612761 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc87801a430 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.908+0000 7fc8777fe700 1 -- 172.21.15.96:0/2220612761 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc878019550 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.908+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88819e810 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.908+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8881a1420 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.909+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc888064cf0 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.909+0000 7fc8777fe700 1 -- 172.21.15.96:0/2220612761 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fc8780197c0 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.909+0000 7fc8777fe700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fc87003d570 0x7fc87003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.909+0000 7fc8777fe700 1 -- 172.21.15.96:0/2220612761 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc8780525f0 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.909+0000 7fc885d9b700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fc87003d570 0x7fc87003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.909+0000 7fc885d9b700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fc87003d570 0x7fc87003fa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc87c006fd0 tx=0x7fc87c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:37.914+0000 7fc8777fe700 1 -- 172.21.15.96:0/2220612761 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc878017e90 con 0x7fc888107c90 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.128+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc888104750 con 0x7fc87003d570 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.133+0000 7fc8777fe700 1 -- 172.21.15.96:0/2220612761 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc888104750 con 0x7fc87003d570 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fc87003d570 msgr2=0x7fc87003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:38.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fc87003d570 0x7fc87003fa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc87c006fd0 tx=0x7fc87c006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 msgr2=0x7fc88819e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc88819e040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc878011670 tx=0x7fc878004280 comp rx=0 tx=0).stop 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 shutdown_connections 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fc87003d570 0x7fc87003fa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 --2- 172.21.15.96:0/2220612761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc888107c90 0x7fc88819e040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 >> 172.21.15.96:0/2220612761 conn(0x7fc8881036a0 msgr2=0x7fc888103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 shutdown_connections 2024-04-25T17:00:38.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:38.135+0000 7fc88cd5b700 1 -- 172.21.15.96:0/2220612761 wait complete. 2024-04-25T17:00:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:38 smithi096 ceph-mon[29131]: [25/Apr/2024:17:00:36] ENGINE Bus STARTING 2024-04-25T17:00:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:38 smithi096 ceph-mon[29131]: [25/Apr/2024:17:00:36] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T17:00:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:38 smithi096 ceph-mon[29131]: [25/Apr/2024:17:00:37] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T17:00:38.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:38 smithi096 ceph-mon[29131]: [25/Apr/2024:17:00:37] ENGINE Bus STARTED 2024-04-25T17:00:38.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:38 smithi096 ceph-mon[29131]: mgrmap e8: x(active, since 2s) 2024-04-25T17:00:38.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:38 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:38.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:38 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:00:39.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T17:00:39.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.049+0000 7f2edf429700 1 Processor -- start 2024-04-25T17:00:39.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.049+0000 7f2edf429700 1 -- start start 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.049+0000 7f2edf429700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed8108c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.049+0000 7f2edf429700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8109270 con 0x7f2ed8108870 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.049+0000 7f2edd1c5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed8108c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.049+0000 7f2edd1c5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed8108c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42524/0 (socket says 172.21.15.96:42524) 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.049+0000 7f2edd1c5700 1 -- 172.21.15.96:0/221221166 learned_addr learned my addr 172.21.15.96:0/221221166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.050+0000 7f2edd1c5700 1 -- 172.21.15.96:0/221221166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed8109a40 con 0x7f2ed8108870 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.050+0000 7f2edd1c5700 1 --2- 172.21.15.96:0/221221166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed8108c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40098a0 tx=0x7f2ed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e0383d439824d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.050+0000 7f2ecffff700 1 -- 172.21.15.96:0/221221166 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ed4004160 con 0x7f2ed8108870 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.050+0000 7f2ecffff700 1 -- 172.21.15.96:0/221221166 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2ed4004300 con 0x7f2ed8108870 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.050+0000 7f2ecffff700 1 -- 172.21.15.96:0/221221166 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ed4011af0 con 0x7f2ed8108870 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 -- 172.21.15.96:0/221221166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 msgr2=0x7f2ed8108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 --2- 172.21.15.96:0/221221166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed8108c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40098a0 tx=0x7f2ed4009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:39.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 -- 172.21.15.96:0/221221166 shutdown_connections 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 --2- 172.21.15.96:0/221221166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed8108c80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 -- 172.21.15.96:0/221221166 >> 172.21.15.96:0/221221166 conn(0x7f2ed8103ed0 msgr2=0x7f2ed8106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 -- 172.21.15.96:0/221221166 shutdown_connections 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 -- 172.21.15.96:0/221221166 wait complete. 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 Processor -- start 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.051+0000 7f2edf429700 1 -- start start 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edf429700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edf429700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed810cdf0 con 0x7f2ed8108870 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edd1c5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edd1c5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42532/0 (socket says 172.21.15.96:42532) 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edd1c5700 1 -- 172.21.15.96:0/3878240607 learned_addr learned my addr 172.21.15.96:0/3878240607 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edd1c5700 1 -- 172.21.15.96:0/3878240607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed819d270 con 0x7f2ed8108870 2024-04-25T17:00:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edd1c5700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed819cd30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4011670 tx=0x7f2ed4004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2ece7fc700 1 -- 172.21.15.96:0/3878240607 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ed40036a0 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2ece7fc700 1 -- 172.21.15.96:0/3878240607 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2ed4003cd0 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2ece7fc700 1 -- 172.21.15.96:0/3878240607 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ed40193c0 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed819d500 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.052+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81a0100 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.053+0000 7f2ece7fc700 1 -- 172.21.15.96:0/3878240607 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f2ed4003840 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.053+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed8064cf0 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.054+0000 7f2ece7fc700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f2ec403d520 0x7f2ec403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.054+0000 7f2ece7fc700 1 -- 172.21.15.96:0/3878240607 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2ed4052610 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.054+0000 7f2edc9c4700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f2ec403d520 0x7f2ec403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.054+0000 7f2edc9c4700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f2ec403d520 0x7f2ec403fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006fd0 tx=0x7f2ec8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.059+0000 7f2ece7fc700 1 -- 172.21.15.96:0/3878240607 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ed401a6c0 con 0x7f2ed8108870 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.256+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2ed8104f70 con 0x7f2ec403d520 2024-04-25T17:00:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.257+0000 7f2ece7fc700 1 -- 172.21.15.96:0/3878240607 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2ed8104f70 con 0x7f2ec403d520 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f2ec403d520 msgr2=0x7f2ec403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f2ec403d520 0x7f2ec403fa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8006fd0 tx=0x7f2ec8006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 msgr2=0x7f2ed819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed819cd30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4011670 tx=0x7f2ed4004280 comp rx=0 tx=0).stop 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 shutdown_connections 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f2ec403d520 0x7f2ec403fa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 --2- 172.21.15.96:0/3878240607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ed8108870 0x7f2ed819cd30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 >> 172.21.15.96:0/3878240607 conn(0x7f2ed8103ed0 msgr2=0x7f2ed8104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 shutdown_connections 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:39.259+0000 7f2edf429700 1 -- 172.21.15.96:0/3878240607 wait complete. 2024-04-25T17:00:39.788 INFO:teuthology.orchestra.run.smithi096.stdout:Generating ssh key... 2024-04-25T17:00:39.884 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:39 smithi096 ceph-mon[29131]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:00:40.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-mon[29131]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:00:41.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: Generating public/private rsa key pair. 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: Your identification has been saved in /tmp/tmp5cle62fc/key. 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: Your public key has been saved in /tmp/tmp5cle62fc/key.pub. 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: The key fingerprint is: 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: SHA256:hibson7JDTGlPvoYMSV+76IEJLslC/0gdc9w6T+U0C8 ceph-1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: The key's randomart image is: 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: +---[RSA 3072]----+ 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: | | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: | . o | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: |.o..= + . | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: |++o* * o o | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: |*+=.= * E . | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: |o*+B.o + . | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: |oo+ B. o | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: |.oo*.. . | 2024-04-25T17:00:41.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: |o++... | 2024-04-25T17:00:41.188 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: +----[SHA256]-----+ 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68ff12700 1 Processor -- start 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68ff12700 1 -- start start 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68ff12700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb688108c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68ff12700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb688109260 con 0x7fb688108860 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68dcae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb688108c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68dcae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb688108c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42548/0 (socket says 172.21.15.96:42548) 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68dcae700 1 -- 172.21.15.96:0/933012020 learned_addr learned my addr 172.21.15.96:0/933012020 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.205+0000 7fb68dcae700 1 -- 172.21.15.96:0/933012020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb688109a30 con 0x7fb688108860 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.206+0000 7fb68dcae700 1 --2- 172.21.15.96:0/933012020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb688108c70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e80 tx=0x7fb68000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37072720914c3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:41.540 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.206+0000 7fb68ccac700 1 -- 172.21.15.96:0/933012020 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680017070 con 0x7fb688108860 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.206+0000 7fb68ccac700 1 -- 172.21.15.96:0/933012020 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb68000b8c0 con 0x7fb688108860 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.206+0000 7fb68ccac700 1 -- 172.21.15.96:0/933012020 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680004490 con 0x7fb688108860 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 -- 172.21.15.96:0/933012020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 msgr2=0x7fb688108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 --2- 172.21.15.96:0/933012020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb688108c70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e80 tx=0x7fb68000b330 comp rx=0 tx=0).stop 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 -- 172.21.15.96:0/933012020 shutdown_connections 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 --2- 172.21.15.96:0/933012020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb688108c70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 -- 172.21.15.96:0/933012020 >> 172.21.15.96:0/933012020 conn(0x7fb688103e80 msgr2=0x7fb688106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 -- 172.21.15.96:0/933012020 shutdown_connections 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 -- 172.21.15.96:0/933012020 wait complete. 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 Processor -- start 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 -- start start 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.207+0000 7fb68ff12700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb68819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68ff12700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb68810cde0 con 0x7fb688108860 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68dcae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb68819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68dcae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb68819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42550/0 (socket says 172.21.15.96:42550) 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68dcae700 1 -- 172.21.15.96:0/490342387 learned_addr learned my addr 172.21.15.96:0/490342387 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68dcae700 1 -- 172.21.15.96:0/490342387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68819e340 con 0x7fb688108860 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68dcae700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb68819de00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb680009fd0 tx=0x7fb68000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb686ffd700 1 -- 172.21.15.96:0/490342387 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680020020 con 0x7fb688108860 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb686ffd700 1 -- 172.21.15.96:0/490342387 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb680012430 con 0x7fb688108860 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68819e5d0 con 0x7fb688108860 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb686ffd700 1 -- 172.21.15.96:0/490342387 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680017040 con 0x7fb688108860 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.208+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6881a11e0 con 0x7fb688108860 2024-04-25T17:00:41.542 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.209+0000 7fb686ffd700 1 -- 172.21.15.96:0/490342387 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fb680023020 con 0x7fb688108860 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.209+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb688064cf0 con 0x7fb688108860 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.209+0000 7fb686ffd700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fb67003d570 0x7fb67003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.209+0000 7fb686ffd700 1 -- 172.21.15.96:0/490342387 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb68000f050 con 0x7fb688108860 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.210+0000 7fb68d4ad700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fb67003d570 0x7fb67003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.210+0000 7fb68d4ad700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fb67003d570 0x7fb67003fa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb67c006fd0 tx=0x7fb67c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.215+0000 7fb686ffd700 1 -- 172.21.15.96:0/490342387 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb68002a020 con 0x7fb688108860 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.409+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb68802e880 con 0x7fb67003d570 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.927+0000 7fb686ffd700 1 -- 172.21.15.96:0/490342387 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb68802e880 con 0x7fb67003d570 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fb67003d570 msgr2=0x7fb67003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fb67003d570 0x7fb67003fa80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb67c006fd0 tx=0x7fb67c006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 msgr2=0x7fb68819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:41.543 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb68819de00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb680009fd0 tx=0x7fb68000bd50 comp rx=0 tx=0).stop 2024-04-25T17:00:41.544 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 shutdown_connections 2024-04-25T17:00:41.544 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fb67003d570 0x7fb67003fa80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:41.544 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 --2- 172.21.15.96:0/490342387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb688108860 0x7fb68819de00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:41.544 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.929+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 >> 172.21.15.96:0/490342387 conn(0x7fb688103e80 msgr2=0x7fb6881943b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:41.544 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.930+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 shutdown_connections 2024-04-25T17:00:41.544 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:40.930+0000 7fb68ff12700 1 -- 172.21.15.96:0/490342387 wait complete. 2024-04-25T17:00:41.909 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:41 smithi096 ceph-mon[29131]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:00:41.909 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:41 smithi096 ceph-mon[29131]: Generating ssh key... 2024-04-25T17:00:41.909 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:41 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:41.909 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:41 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:42.771 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0MrRAlSpiTTpC0JSNXwgHqO5EaQtn/79plIXYVqg7XcK/mTyLnh+8tDjhrGomj3LkeUGYVUq6PzTyfNPq2Ny9/OonpfAgYDwU07df6H0NnjWatHn4Jom2eSODf5eQ6rY1z7+7+B9z+n9aJFMT+4CkwQ9VNmtOTouUP5Pl3+yWCXxe6FtMRbpDfxXYy+lsaAVROxS0NLpzoaGEMd51vkQ1J5oS0fdPdodKL34gLuc1Gu8PtPKmRxYNO97uz2Njs1CTILOqGnC9WcwVrCOi7z/IQNGhGMvZYBm5PrmRad705WBNUCQEzNYRct0ZeFv50jCS65I9c0c11SRki9pEWwSDZKhbk05uUXRrph3sH0Oc2gSS99n5Hi0YeJuy1jlx4sI8pZzjdFZwLcoB++V32XZIRVNQ7IFflu0C2xxe9L1FSJbxtdLCv4p9bgiXualJ6f/Ls4y1kKvlQO03pfK80YUPa4G0ia1XD5aJEvIpkUZhpcMXr6G3M0cU2JU8UQZUXfM= ceph-1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.026+0000 7f7ed55fa700 1 Processor -- start 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.026+0000 7f7ed55fa700 1 -- start start 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.026+0000 7f7ed55fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed0108c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.026+0000 7f7ed55fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0109240 con 0x7f7ed0108840 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.026+0000 7f7eceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed0108c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.026+0000 7f7eceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed0108c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42566/0 (socket says 172.21.15.96:42566) 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.026+0000 7f7eceffd700 1 -- 172.21.15.96:0/4150861418 learned_addr learned my addr 172.21.15.96:0/4150861418 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.027+0000 7f7eceffd700 1 -- 172.21.15.96:0/4150861418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed0109a10 con 0x7f7ed0108840 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.027+0000 7f7eceffd700 1 --2- 172.21.15.96:0/4150861418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed0108c50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098a0 tx=0x7f7ec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc222ebd1ff392c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.027+0000 7f7ece7fc700 1 -- 172.21.15.96:0/4150861418 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0004160 con 0x7f7ed0108840 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.027+0000 7f7ece7fc700 1 -- 172.21.15.96:0/4150861418 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ec0004300 con 0x7f7ed0108840 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.027+0000 7f7ece7fc700 1 -- 172.21.15.96:0/4150861418 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0011af0 con 0x7f7ed0108840 2024-04-25T17:00:42.772 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 -- 172.21.15.96:0/4150861418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 msgr2=0x7f7ed0108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 --2- 172.21.15.96:0/4150861418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed0108c50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098a0 tx=0x7f7ec0009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 -- 172.21.15.96:0/4150861418 shutdown_connections 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 --2- 172.21.15.96:0/4150861418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed0108c50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 -- 172.21.15.96:0/4150861418 >> 172.21.15.96:0/4150861418 conn(0x7f7ed0103f00 msgr2=0x7f7ed0106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 -- 172.21.15.96:0/4150861418 shutdown_connections 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 -- 172.21.15.96:0/4150861418 wait complete. 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.028+0000 7f7ed55fa700 1 Processor -- start 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7ed55fa700 1 -- start start 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7ed55fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed007c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7ed55fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010cdc0 con 0x7f7ed0108840 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7eceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed007c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7eceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed007c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42572/0 (socket says 172.21.15.96:42572) 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7eceffd700 1 -- 172.21.15.96:0/1767731131 learned_addr learned my addr 172.21.15.96:0/1767731131 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7eceffd700 1 -- 172.21.15.96:0/1767731131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed007ce90 con 0x7f7ed0108840 2024-04-25T17:00:42.773 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.029+0000 7f7eceffd700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed007c950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011670 tx=0x7f7ec0004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.030+0000 7f7eccff9700 1 -- 172.21.15.96:0/1767731131 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec00036a0 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.030+0000 7f7eccff9700 1 -- 172.21.15.96:0/1767731131 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ec0003cd0 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.030+0000 7f7eccff9700 1 -- 172.21.15.96:0/1767731131 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec00193c0 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.030+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01a0be0 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.030+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a1060 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.030+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0064cf0 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.031+0000 7f7eccff9700 1 -- 172.21.15.96:0/1767731131 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f7ec0019630 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.031+0000 7f7eccff9700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f7ebc03d570 0x7f7ebc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.031+0000 7f7eccff9700 1 -- 172.21.15.96:0/1767731131 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7ec00524d0 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.031+0000 7f7ec67fc700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f7ebc03d570 0x7f7ebc03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.031+0000 7f7ec67fc700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f7ebc03d570 0x7f7ebc03fa80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8006fd0 tx=0x7f7eb8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.036+0000 7f7eccff9700 1 -- 172.21.15.96:0/1767731131 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ec0017e90 con 0x7f7ed0108840 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.233+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7ed0105fd0 con 0x7f7ebc03d570 2024-04-25T17:00:42.774 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.234+0000 7f7eccff9700 1 -- 172.21.15.96:0/1767731131 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7ed0105fd0 con 0x7f7ebc03d570 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f7ebc03d570 msgr2=0x7f7ebc03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f7ebc03d570 0x7f7ebc03fa80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8006fd0 tx=0x7f7eb8006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 msgr2=0x7f7ed007c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed007c950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011670 tx=0x7f7ec0004280 comp rx=0 tx=0).stop 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 shutdown_connections 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f7ebc03d570 0x7f7ebc03fa80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 --2- 172.21.15.96:0/1767731131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ed0108840 0x7f7ed007c950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 >> 172.21.15.96:0/1767731131 conn(0x7f7ed0103f00 msgr2=0x7f7ed0105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 shutdown_connections 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:42.236+0000 7f7ed55fa700 1 -- 172.21.15.96:0/1767731131 wait complete. 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T17:00:42.775 INFO:teuthology.orchestra.run.smithi096.stdout:Adding host smithi096... 2024-04-25T17:00:43.124 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:42 smithi096 ceph-mon[29131]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:00:44.053 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:43 smithi096 ceph-mon[29131]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:00:45.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:44 smithi096 ceph-mon[29131]: Deploying cephadm binary to smithi096 2024-04-25T17:00:48.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Added host 'smithi096' with addr '172.21.15.96' 2024-04-25T17:00:48.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.240+0000 7f4d58c51700 1 Processor -- start 2024-04-25T17:00:48.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.240+0000 7f4d58c51700 1 -- start start 2024-04-25T17:00:48.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.241+0000 7f4d58c51700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5410a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:48.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.241+0000 7f4d58c51700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410a830 con 0x7f4d54109e10 2024-04-25T17:00:48.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.241+0000 7f4d5259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5410a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.241+0000 7f4d5259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5410a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42578/0 (socket says 172.21.15.96:42578) 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.241+0000 7f4d5259c700 1 -- 172.21.15.96:0/2309776579 learned_addr learned my addr 172.21.15.96:0/2309776579 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.241+0000 7f4d5259c700 1 -- 172.21.15.96:0/2309776579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d54108a40 con 0x7f4d54109e10 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.241+0000 7f4d5259c700 1 --2- 172.21.15.96:0/2309776579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5410a220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4d440098a0 tx=0x7f4d44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdbe35d3cea6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d5159a700 1 -- 172.21.15.96:0/2309776579 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d44004160 con 0x7f4d54109e10 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d5159a700 1 -- 172.21.15.96:0/2309776579 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4d44004300 con 0x7f4d54109e10 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d5159a700 1 -- 172.21.15.96:0/2309776579 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d44011af0 con 0x7f4d54109e10 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d58c51700 1 -- 172.21.15.96:0/2309776579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 msgr2=0x7f4d5410a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d58c51700 1 --2- 172.21.15.96:0/2309776579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5410a220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4d440098a0 tx=0x7f4d44009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d58c51700 1 -- 172.21.15.96:0/2309776579 shutdown_connections 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d58c51700 1 --2- 172.21.15.96:0/2309776579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5410a220 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d58c51700 1 -- 172.21.15.96:0/2309776579 >> 172.21.15.96:0/2309776579 conn(0x7f4d541036c0 msgr2=0x7f4d54105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d58c51700 1 -- 172.21.15.96:0/2309776579 shutdown_connections 2024-04-25T17:00:48.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.242+0000 7f4d58c51700 1 -- 172.21.15.96:0/2309776579 wait complete. 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.243+0000 7f4d58c51700 1 Processor -- start 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.243+0000 7f4d58c51700 1 -- start start 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.243+0000 7f4d58c51700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.243+0000 7f4d58c51700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410ce00 con 0x7f4d54109e10 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d5259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d5259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42582/0 (socket says 172.21.15.96:42582) 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d5259c700 1 -- 172.21.15.96:0/1803363966 learned_addr learned my addr 172.21.15.96:0/1803363966 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d5259c700 1 -- 172.21.15.96:0/1803363966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5419d1a0 con 0x7f4d54109e10 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d5259c700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5419cc60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009fb0 tx=0x7f4d44011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d437fe700 1 -- 172.21.15.96:0/1803363966 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d440036a0 con 0x7f4d54109e10 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d437fe700 1 -- 172.21.15.96:0/1803363966 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4d4401a430 con 0x7f4d54109e10 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5419d430 con 0x7f4d54109e10 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d437fe700 1 -- 172.21.15.96:0/1803363966 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d44019550 con 0x7f4d54109e10 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.244+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541a0040 con 0x7f4d54109e10 2024-04-25T17:00:48.056 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.245+0000 7f4d437fe700 1 -- 172.21.15.96:0/1803363966 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f4d44019800 con 0x7f4d54109e10 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.245+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d54064cf0 con 0x7f4d54109e10 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.246+0000 7f4d437fe700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f4d3c03d570 0x7f4d3c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.246+0000 7f4d437fe700 1 -- 172.21.15.96:0/1803363966 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d44052660 con 0x7f4d54109e10 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.246+0000 7f4d51d9b700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f4d3c03d570 0x7f4d3c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.246+0000 7f4d51d9b700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f4d3c03d570 0x7f4d3c03fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4d48006fd0 tx=0x7f4d48006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.251+0000 7f4d437fe700 1 -- 172.21.15.96:0/1803363966 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d44017e90 con 0x7f4d54109e10 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:43.453+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}) -- 0x7f4d54105740 con 0x7f4d3c03d570 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.514+0000 7f4d437fe700 1 -- 172.21.15.96:0/1803363966 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4d54105740 con 0x7f4d3c03d570 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f4d3c03d570 msgr2=0x7f4d3c03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f4d3c03d570 0x7f4d3c03fa80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4d48006fd0 tx=0x7f4d48006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 msgr2=0x7f4d5419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5419cc60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009fb0 tx=0x7f4d44011940 comp rx=0 tx=0).stop 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 shutdown_connections 2024-04-25T17:00:48.057 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f4d3c03d570 0x7f4d3c03fa80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:48.058 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 --2- 172.21.15.96:0/1803363966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d54109e10 0x7f4d5419cc60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:48.058 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 >> 172.21.15.96:0/1803363966 conn(0x7f4d541036c0 msgr2=0x7f4d54105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:48.058 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 shutdown_connections 2024-04-25T17:00:48.058 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:47.517+0000 7f4d58c51700 1 -- 172.21.15.96:0/1803363966 wait complete. 2024-04-25T17:00:48.058 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mon service... 2024-04-25T17:00:48.784 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:48 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:48.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:48 smithi096 ceph-mon[29131]: Added host smithi096 2024-04-25T17:00:48.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:48 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.557+0000 7efd1a822700 1 Processor -- start 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.557+0000 7efd1a822700 1 -- start start 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.557+0000 7efd1a822700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd14108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.557+0000 7efd1a822700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd141092a0 con 0x7efd141088a0 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd13fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd14108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd13fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd14108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32924/0 (socket says 172.21.15.96:32924) 2024-04-25T17:00:49.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd13fff700 1 -- 172.21.15.96:0/1195199466 learned_addr learned my addr 172.21.15.96:0/1195199466 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd13fff700 1 -- 172.21.15.96:0/1195199466 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd14109a70 con 0x7efd141088a0 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd13fff700 1 --2- 172.21.15.96:0/1195199466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd14108cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efd040098a0 tx=0x7efd04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb8ddf20705bc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd12ffd700 1 -- 172.21.15.96:0/1195199466 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd04004160 con 0x7efd141088a0 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd12ffd700 1 -- 172.21.15.96:0/1195199466 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd04004300 con 0x7efd141088a0 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.558+0000 7efd12ffd700 1 -- 172.21.15.96:0/1195199466 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd04011af0 con 0x7efd141088a0 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.559+0000 7efd1a822700 1 -- 172.21.15.96:0/1195199466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 msgr2=0x7efd14108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.559+0000 7efd1a822700 1 --2- 172.21.15.96:0/1195199466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd14108cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efd040098a0 tx=0x7efd04009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.559+0000 7efd1a822700 1 -- 172.21.15.96:0/1195199466 shutdown_connections 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.559+0000 7efd1a822700 1 --2- 172.21.15.96:0/1195199466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd14108cb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.559+0000 7efd1a822700 1 -- 172.21.15.96:0/1195199466 >> 172.21.15.96:0/1195199466 conn(0x7efd14103f20 msgr2=0x7efd14106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.559+0000 7efd1a822700 1 -- 172.21.15.96:0/1195199466 shutdown_connections 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.559+0000 7efd1a822700 1 -- 172.21.15.96:0/1195199466 wait complete. 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd1a822700 1 Processor -- start 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd1a822700 1 -- start start 2024-04-25T17:00:49.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd1a822700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd1407d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd1a822700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd1410ce20 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd13fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd1407d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd13fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd1407d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32928/0 (socket says 172.21.15.96:32928) 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd13fff700 1 -- 172.21.15.96:0/1752851427 learned_addr learned my addr 172.21.15.96:0/1752851427 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.560+0000 7efd13fff700 1 -- 172.21.15.96:0/1752851427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1407d740 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.561+0000 7efd13fff700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd1407d200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efd04011670 tx=0x7efd04004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.561+0000 7efd117fa700 1 -- 172.21.15.96:0/1752851427 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd040036a0 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.561+0000 7efd117fa700 1 -- 172.21.15.96:0/1752851427 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd04003cd0 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.561+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd141a0bf0 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.561+0000 7efd117fa700 1 -- 172.21.15.96:0/1752851427 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd040193c0 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.561+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141a1070 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.562+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd14064cf0 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.562+0000 7efd117fa700 1 -- 172.21.15.96:0/1752851427 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7efd04019680 con 0x7efd141088a0 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.562+0000 7efd117fa700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7efcfc03d520 0x7efcfc03fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:49.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.562+0000 7efd117fa700 1 -- 172.21.15.96:0/1752851427 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efd04052740 con 0x7efd141088a0 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.562+0000 7efd137fe700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7efcfc03d520 0x7efcfc03fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.563+0000 7efd137fe700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7efcfc03d520 0x7efcfc03fa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efd08006fd0 tx=0x7efd08006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.567+0000 7efd117fa700 1 -- 172.21.15.96:0/1752851427 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd0401a430 con 0x7efd141088a0 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.786+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7efd14104f80 con 0x7efcfc03d520 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.794+0000 7efd117fa700 1 -- 172.21.15.96:0/1752851427 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efd14104f80 con 0x7efcfc03d520 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.796+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7efcfc03d520 msgr2=0x7efcfc03fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.796+0000 7efd1a822700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7efcfc03d520 0x7efcfc03fa30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efd08006fd0 tx=0x7efd08006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.796+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 msgr2=0x7efd1407d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.796+0000 7efd1a822700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd1407d200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efd04011670 tx=0x7efd04004280 comp rx=0 tx=0).stop 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.796+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 shutdown_connections 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.796+0000 7efd1a822700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7efcfc03d520 0x7efcfc03fa30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.796+0000 7efd1a822700 1 --2- 172.21.15.96:0/1752851427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd141088a0 0x7efd1407d200 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.797+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 >> 172.21.15.96:0/1752851427 conn(0x7efd14103f20 msgr2=0x7efd14104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:49.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.797+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 shutdown_connections 2024-04-25T17:00:49.487 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:48.797+0000 7efd1a822700 1 -- 172.21.15.96:0/1752851427 wait complete. 2024-04-25T17:00:49.487 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mgr service... 2024-04-25T17:00:50.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:49 smithi096 ceph-mon[29131]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:00:50.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:49 smithi096 ceph-mon[29131]: Saving service mon spec with placement count:5 2024-04-25T17:00:50.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:49 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.655+0000 7f347d22e700 1 Processor -- start 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f347d22e700 1 -- start start 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f347d22e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f347810c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f347d22e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f347807ce40 con 0x7f347810a460 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f347810c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f347810c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32942/0 (socket says 172.21.15.96:32942) 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f3476d9d700 1 -- 172.21.15.96:0/3460799904 learned_addr learned my addr 172.21.15.96:0/3460799904 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f3476d9d700 1 -- 172.21.15.96:0/3460799904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347810cdc0 con 0x7f347810a460 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.656+0000 7f3476d9d700 1 --2- 172.21.15.96:0/3460799904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f347810c880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34680098a0 tx=0x7f3468009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69df701ef0f7face server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:51.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f3475d9b700 1 -- 172.21.15.96:0/3460799904 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3468004160 con 0x7f347810a460 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f3475d9b700 1 -- 172.21.15.96:0/3460799904 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3468004300 con 0x7f347810a460 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f3475d9b700 1 -- 172.21.15.96:0/3460799904 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3468011a20 con 0x7f347810a460 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f347d22e700 1 -- 172.21.15.96:0/3460799904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 msgr2=0x7f347810c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f347d22e700 1 --2- 172.21.15.96:0/3460799904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f347810c880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34680098a0 tx=0x7f3468009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f347d22e700 1 -- 172.21.15.96:0/3460799904 shutdown_connections 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f347d22e700 1 --2- 172.21.15.96:0/3460799904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f347810c880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f347d22e700 1 -- 172.21.15.96:0/3460799904 >> 172.21.15.96:0/3460799904 conn(0x7f3478103f00 msgr2=0x7f3478106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.657+0000 7f347d22e700 1 -- 172.21.15.96:0/3460799904 shutdown_connections 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.658+0000 7f347d22e700 1 -- 172.21.15.96:0/3460799904 wait complete. 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.658+0000 7f347d22e700 1 Processor -- start 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.658+0000 7f347d22e700 1 -- start start 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f347d22e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f3478109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f347d22e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f347810dd60 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f3478109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f3476d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f3478109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32956/0 (socket says 172.21.15.96:32956) 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f3476d9d700 1 -- 172.21.15.96:0/6978073 learned_addr learned my addr 172.21.15.96:0/6978073 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f3476d9d700 1 -- 172.21.15.96:0/6978073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347810a120 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f3476d9d700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f3478109be0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3468004660 tx=0x7f3468011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f345ffff700 1 -- 172.21.15.96:0/6978073 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3468003a60 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3478108370 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.659+0000 7f345ffff700 1 -- 172.21.15.96:0/6978073 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346801b8f0 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.660+0000 7f345ffff700 1 -- 172.21.15.96:0/6978073 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3468023890 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.660+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3478108850 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.661+0000 7f345ffff700 1 -- 172.21.15.96:0/6978073 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f346802c3f0 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.661+0000 7f345ffff700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f346003d520 0x7f346003fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.661+0000 7f345ffff700 1 -- 172.21.15.96:0/6978073 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3468052f50 con 0x7f347810a460 2024-04-25T17:00:51.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.661+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3478064cf0 con 0x7f347810a460 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.661+0000 7f347659c700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f346003d520 0x7f346003fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.661+0000 7f347659c700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f346003d520 0x7f346003fa30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f346c006fd0 tx=0x7f346c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.666+0000 7f345ffff700 1 -- 172.21.15.96:0/6978073 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f346801f030 con 0x7f347810a460 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.894+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3478108ed0 con 0x7f346003d520 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.902+0000 7f345ffff700 1 -- 172.21.15.96:0/6978073 <== mgr.14116 v2:172.21.15.96:6800/1771523969 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3478108ed0 con 0x7f346003d520 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f346003d520 msgr2=0x7f346003fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f346003d520 0x7f346003fa30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f346c006fd0 tx=0x7f346c006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 msgr2=0x7f3478109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f3478109be0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3468004660 tx=0x7f3468011aa0 comp rx=0 tx=0).stop 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 shutdown_connections 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f346003d520 0x7f346003fa30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 --2- 172.21.15.96:0/6978073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f347810a460 0x7f3478109be0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 >> 172.21.15.96:0/6978073 conn(0x7f3478103f00 msgr2=0x7f3478105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 shutdown_connections 2024-04-25T17:00:51.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:50.905+0000 7f347d22e700 1 -- 172.21.15.96:0/6978073 wait complete. 2024-04-25T17:00:51.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:51 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:51.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:51 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:51.585 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:51 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:52.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:52 smithi096 ceph-mon[29131]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:00:52.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:52 smithi096 ceph-mon[29131]: Saving service mgr spec with placement count:2 2024-04-25T17:00:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:52 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2085630537' entity='client.admin' 2024-04-25T17:00:53.270 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.903+0000 7f811a18a700 1 Processor -- start 2024-04-25T17:00:53.270 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.903+0000 7f811a18a700 1 -- start start 2024-04-25T17:00:53.270 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f811a18a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f81141086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f811a18a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f811407ce00 con 0x7f81141082c0 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81137fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f81141086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81137fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f81141086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32972/0 (socket says 172.21.15.96:32972) 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81137fe700 1 -- 172.21.15.96:0/1177971490 learned_addr learned my addr 172.21.15.96:0/1177971490 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81137fe700 1 -- 172.21.15.96:0/1177971490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8114108c10 con 0x7f81141082c0 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81137fe700 1 --2- 172.21.15.96:0/1177971490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f81141086d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8104009e80 tx=0x7f810400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24d72132bc67d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81127fc700 1 -- 172.21.15.96:0/1177971490 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104017070 con 0x7f81141082c0 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81127fc700 1 -- 172.21.15.96:0/1177971490 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810400b8c0 con 0x7f81141082c0 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.904+0000 7f81127fc700 1 -- 172.21.15.96:0/1177971490 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104004490 con 0x7f81141082c0 2024-04-25T17:00:53.271 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.905+0000 7f811a18a700 1 -- 172.21.15.96:0/1177971490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 msgr2=0x7f81141086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:53.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.905+0000 7f811a18a700 1 --2- 172.21.15.96:0/1177971490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f81141086d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8104009e80 tx=0x7f810400b330 comp rx=0 tx=0).stop 2024-04-25T17:00:53.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.905+0000 7f811a18a700 1 -- 172.21.15.96:0/1177971490 shutdown_connections 2024-04-25T17:00:53.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.905+0000 7f811a18a700 1 --2- 172.21.15.96:0/1177971490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f81141086d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:53.273 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.905+0000 7f811a18a700 1 -- 172.21.15.96:0/1177971490 >> 172.21.15.96:0/1177971490 conn(0x7f8114103f50 msgr2=0x7f81141063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.905+0000 7f811a18a700 1 -- 172.21.15.96:0/1177971490 shutdown_connections 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.905+0000 7f811a18a700 1 -- 172.21.15.96:0/1177971490 wait complete. 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.906+0000 7f811a18a700 1 Processor -- start 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.906+0000 7f811a18a700 1 -- start start 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.906+0000 7f811a18a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f811419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.906+0000 7f811a18a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8114109cd0 con 0x7f81141082c0 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.906+0000 7f81137fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f811419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.906+0000 7f81137fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f811419ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32986/0 (socket says 172.21.15.96:32986) 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.906+0000 7f81137fe700 1 -- 172.21.15.96:0/2085630537 learned_addr learned my addr 172.21.15.96:0/2085630537 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.907+0000 7f81137fe700 1 -- 172.21.15.96:0/2085630537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141a0490 con 0x7f81141082c0 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.907+0000 7f81137fe700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f811419ff50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8104009e50 tx=0x7f810400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.907+0000 7f8110ff9700 1 -- 172.21.15.96:0/2085630537 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104020020 con 0x7f81141082c0 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.907+0000 7f8110ff9700 1 -- 172.21.15.96:0/2085630537 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8104012430 con 0x7f81141082c0 2024-04-25T17:00:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.907+0000 7f8110ff9700 1 -- 172.21.15.96:0/2085630537 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104017040 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.907+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81141a0720 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.907+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a3330 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.908+0000 7f8110ff9700 1 -- 172.21.15.96:0/2085630537 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f8104023020 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.908+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8114064cf0 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.908+0000 7f8110ff9700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f80fc03d570 0x7f80fc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.908+0000 7f8110ff9700 1 -- 172.21.15.96:0/2085630537 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f810400f050 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.908+0000 7f8112ffd700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f80fc03d570 0x7f80fc03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.909+0000 7f8112ffd700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f80fc03d570 0x7f80fc03fa80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8108006fd0 tx=0x7f8108006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:51.913+0000 7f8110ff9700 1 -- 172.21.15.96:0/2085630537 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f810402a020 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.134+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f811402e7e0 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.137+0000 7f8110ff9700 1 -- 172.21.15.96:0/2085630537 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f81040102f0 con 0x7f81141082c0 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f80fc03d570 msgr2=0x7f80fc03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:53.275 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f80fc03d570 0x7f80fc03fa80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8108006fd0 tx=0x7f8108006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 msgr2=0x7f811419ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f811419ff50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8104009e50 tx=0x7f810400bd50 comp rx=0 tx=0).stop 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 shutdown_connections 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f80fc03d570 0x7f80fc03fa80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 --2- 172.21.15.96:0/2085630537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81141082c0 0x7f811419ff50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.139+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 >> 172.21.15.96:0/2085630537 conn(0x7f8114103f50 msgr2=0x7f8114193ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.140+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 shutdown_connections 2024-04-25T17:00:53.276 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:52.140+0000 7f811a18a700 1 -- 172.21.15.96:0/2085630537 wait complete. 2024-04-25T17:00:54.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:53 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:54.476 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d9d264700 1 Processor -- start 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d9d264700 1 -- start start 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d9d264700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d98108690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d9d264700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9807d6d0 con 0x7f9d98108280 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d98108690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d98108690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32998/0 (socket says 172.21.15.96:32998) 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d96ffd700 1 -- 172.21.15.96:0/3849323810 learned_addr learned my addr 172.21.15.96:0/3849323810 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.742+0000 7f9d96ffd700 1 -- 172.21.15.96:0/3849323810 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d98108bd0 con 0x7f9d98108280 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.743+0000 7f9d96ffd700 1 --2- 172.21.15.96:0/3849323810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d98108690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098a0 tx=0x7f9d88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c882d93aebdbe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.743+0000 7f9d95ffb700 1 -- 172.21.15.96:0/3849323810 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d88004160 con 0x7f9d98108280 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.743+0000 7f9d95ffb700 1 -- 172.21.15.96:0/3849323810 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d88004300 con 0x7f9d98108280 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.743+0000 7f9d95ffb700 1 -- 172.21.15.96:0/3849323810 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d88011a20 con 0x7f9d98108280 2024-04-25T17:00:54.477 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.744+0000 7f9d9d264700 1 -- 172.21.15.96:0/3849323810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 msgr2=0x7f9d98108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.744+0000 7f9d9d264700 1 --2- 172.21.15.96:0/3849323810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d98108690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098a0 tx=0x7f9d88009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.744+0000 7f9d9d264700 1 -- 172.21.15.96:0/3849323810 shutdown_connections 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.744+0000 7f9d9d264700 1 --2- 172.21.15.96:0/3849323810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d98108690 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.744+0000 7f9d9d264700 1 -- 172.21.15.96:0/3849323810 >> 172.21.15.96:0/3849323810 conn(0x7f9d98103ed0 msgr2=0x7f9d98106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.744+0000 7f9d9d264700 1 -- 172.21.15.96:0/3849323810 shutdown_connections 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.744+0000 7f9d9d264700 1 -- 172.21.15.96:0/3849323810 wait complete. 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d9d264700 1 Processor -- start 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d9d264700 1 -- start start 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d9d264700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d9819ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d9d264700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98109c90 con 0x7f9d98108280 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d9819ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d96ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d9819ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33010/0 (socket says 172.21.15.96:33010) 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d96ffd700 1 -- 172.21.15.96:0/1381305060 learned_addr learned my addr 172.21.15.96:0/1381305060 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:54.478 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.745+0000 7f9d96ffd700 1 -- 172.21.15.96:0/1381305060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d981a0480 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.746+0000 7f9d96ffd700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d9819ff40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d88004660 tx=0x7f9d88011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.746+0000 7f9d8ffff700 1 -- 172.21.15.96:0/1381305060 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d88003a60 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.746+0000 7f9d8ffff700 1 -- 172.21.15.96:0/1381305060 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8801b8f0 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.746+0000 7f9d8ffff700 1 -- 172.21.15.96:0/1381305060 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d88023890 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.746+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981a0710 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.746+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a3320 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.747+0000 7f9d8ffff700 1 -- 172.21.15.96:0/1381305060 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f9d8802c3f0 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.747+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d98064cf0 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.747+0000 7f9d8ffff700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f9d8403d520 0x7f9d8403fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.747+0000 7f9d8ffff700 1 -- 172.21.15.96:0/1381305060 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9d88052d80 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.747+0000 7f9d967fc700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f9d8403d520 0x7f9d8403fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.748+0000 7f9d967fc700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f9d8403d520 0x7f9d8403fa30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9d80006fd0 tx=0x7f9d80006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.757+0000 7f9d8ffff700 1 -- 172.21.15.96:0/1381305060 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d8801f030 con 0x7f9d98108280 2024-04-25T17:00:54.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.965+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f9d9802e850 con 0x7f9d98108280 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.969+0000 7f9d8ffff700 1 -- 172.21.15.96:0/1381305060 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f9d880172b0 con 0x7f9d98108280 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f9d8403d520 msgr2=0x7f9d8403fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f9d8403d520 0x7f9d8403fa30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9d80006fd0 tx=0x7f9d80006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 msgr2=0x7f9d9819ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d9819ff40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d88004660 tx=0x7f9d88011aa0 comp rx=0 tx=0).stop 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 shutdown_connections 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f9d8403d520 0x7f9d8403fa30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 --2- 172.21.15.96:0/1381305060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d98108280 0x7f9d9819ff40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 >> 172.21.15.96:0/1381305060 conn(0x7f9d98103ed0 msgr2=0x7f9d98193f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 shutdown_connections 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:53.971+0000 7f9d9d264700 1 -- 172.21.15.96:0/1381305060 wait complete. 2024-04-25T17:00:54.480 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling the dashboard module... 2024-04-25T17:00:55.122 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:54 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1381305060' entity='client.admin' 2024-04-25T17:00:55.122 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:54 smithi096 ceph-mon[29131]: from='mgr.14116 172.21.15.96:0/2713930671' entity='mgr.x' 2024-04-25T17:00:56.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:55 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/895945924' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T17:00:56.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:55 smithi096 ceph-mon[29131]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:00:56.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: ignoring --setuser ceph since I am not root 2024-04-25T17:00:56.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: ignoring --setgroup ceph since I am not root 2024-04-25T17:00:56.488 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:56.191+0000 7ff8f10a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:00:56.489 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:56.385+0000 7ff8f10a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:00:56.489 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:56.489+0000 7ff8f10a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:00:56.593 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.863+0000 7ff9d9372700 1 Processor -- start 2024-04-25T17:00:56.598 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.863+0000 7ff9d9372700 1 -- start start 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.863+0000 7ff9d9372700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d4108110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.863+0000 7ff9d9372700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d4108700 con 0x7ff9d4107d00 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d4108110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d4108110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33012/0 (socket says 172.21.15.96:33012) 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d2ffd700 1 -- 172.21.15.96:0/1083225329 learned_addr learned my addr 172.21.15.96:0/1083225329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d2ffd700 1 -- 172.21.15.96:0/1083225329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d4108880 con 0x7ff9d4107d00 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d2ffd700 1 --2- 172.21.15.96:0/1083225329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d4108110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40098a0 tx=0x7ff9c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ad6319ddd5fff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d1ffb700 1 -- 172.21.15.96:0/1083225329 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c4004160 con 0x7ff9d4107d00 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d1ffb700 1 -- 172.21.15.96:0/1083225329 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c4004300 con 0x7ff9d4107d00 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.864+0000 7ff9d1ffb700 1 -- 172.21.15.96:0/1083225329 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c4011a20 con 0x7ff9d4107d00 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.865+0000 7ff9d9372700 1 -- 172.21.15.96:0/1083225329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 msgr2=0x7ff9d4108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:56.599 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.865+0000 7ff9d9372700 1 --2- 172.21.15.96:0/1083225329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d4108110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40098a0 tx=0x7ff9c4009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.865+0000 7ff9d9372700 1 -- 172.21.15.96:0/1083225329 shutdown_connections 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.865+0000 7ff9d9372700 1 --2- 172.21.15.96:0/1083225329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d4108110 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.865+0000 7ff9d9372700 1 -- 172.21.15.96:0/1083225329 >> 172.21.15.96:0/1083225329 conn(0x7ff9d41036c0 msgr2=0x7ff9d4105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.865+0000 7ff9d9372700 1 -- 172.21.15.96:0/1083225329 shutdown_connections 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.865+0000 7ff9d9372700 1 -- 172.21.15.96:0/1083225329 wait complete. 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.866+0000 7ff9d9372700 1 Processor -- start 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d9372700 1 -- start start 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d9372700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d410a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d9372700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d410ddc0 con 0x7ff9d4107d00 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d410a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d410a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33016/0 (socket says 172.21.15.96:33016) 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d2ffd700 1 -- 172.21.15.96:0/895945924 learned_addr learned my addr 172.21.15.96:0/895945924 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d2ffd700 1 -- 172.21.15.96:0/895945924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d410aa50 con 0x7ff9d4107d00 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9d2ffd700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d410a510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4003f00 tx=0x7ff9c4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:56.600 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.867+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c4003aa0 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.868+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c401b8f0 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.868+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d4108ca0 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.868+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d4109180 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.868+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c4023890 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.869+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7ff9c402c3f0 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.869+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d4064cf0 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.869+0000 7ff9bbfff700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7ff9bc03d5c0 0x7ff9bc03fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.869+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff9c4052c70 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.869+0000 7ff9d27fc700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7ff9bc03d5c0 0x7ff9bc03fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.870+0000 7ff9d27fc700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7ff9bc03d5c0 0x7ff9bc03fad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006fd0 tx=0x7ff9c8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:54.875+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9c401f030 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.147+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff9d4109870 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.972+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff9c401bda0 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.974+0000 7ff9bbfff700 1 -- 172.21.15.96:0/895945924 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7ff9c40193d0 con 0x7ff9d4107d00 2024-04-25T17:00:56.601 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7ff9bc03d5c0 msgr2=0x7ff9bc03fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7ff9bc03d5c0 0x7ff9bc03fad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006fd0 tx=0x7ff9c8006e40 comp rx=0 tx=0).stop 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 msgr2=0x7ff9d410a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d410a510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4003f00 tx=0x7ff9c4011aa0 comp rx=0 tx=0).stop 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 shutdown_connections 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7ff9bc03d5c0 0x7ff9bc03fad0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 --2- 172.21.15.96:0/895945924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d4107d00 0x7ff9d410a510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 >> 172.21.15.96:0/895945924 conn(0x7ff9d41036c0 msgr2=0x7ff9d41050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.975+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 shutdown_connections 2024-04-25T17:00:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:55.976+0000 7ff9d9372700 1 -- 172.21.15.96:0/895945924 wait complete. 2024-04-25T17:00:57.059 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/895945924' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T17:00:57.059 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-mon[29131]: mgrmap e9: x(active, since 20s) 2024-04-25T17:00:57.059 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:56.810+0000 7ff8f10a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:00:57.059 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:56.900+0000 7ff8f10a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:00:57.981 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.058+0000 7fa449d89700 1 Processor -- start 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.058+0000 7fa449d89700 1 -- start start 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.058+0000 7fa449d89700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44410b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.058+0000 7fa449d89700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410b960 con 0x7fa44410af80 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44410b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44410b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33044/0 (socket says 172.21.15.96:33044) 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa4437fe700 1 -- 172.21.15.96:0/3412658024 learned_addr learned my addr 172.21.15.96:0/3412658024 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:57.982 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa4437fe700 1 -- 172.21.15.96:0/3412658024 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44410bae0 con 0x7fa44410af80 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa4437fe700 1 --2- 172.21.15.96:0/3412658024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44410b390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa4340098a0 tx=0x7fa434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8803794218102a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa442ffd700 1 -- 172.21.15.96:0/3412658024 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa434004160 con 0x7fa44410af80 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa442ffd700 1 -- 172.21.15.96:0/3412658024 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa434004300 con 0x7fa44410af80 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.059+0000 7fa442ffd700 1 -- 172.21.15.96:0/3412658024 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa434011a20 con 0x7fa44410af80 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.060+0000 7fa449d89700 1 -- 172.21.15.96:0/3412658024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 msgr2=0x7fa44410b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.060+0000 7fa449d89700 1 --2- 172.21.15.96:0/3412658024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44410b390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa4340098a0 tx=0x7fa434009bb0 comp rx=0 tx=0).stop 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.060+0000 7fa449d89700 1 -- 172.21.15.96:0/3412658024 shutdown_connections 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.060+0000 7fa449d89700 1 --2- 172.21.15.96:0/3412658024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44410b390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.060+0000 7fa449d89700 1 -- 172.21.15.96:0/3412658024 >> 172.21.15.96:0/3412658024 conn(0x7fa4441036c0 msgr2=0x7fa444105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.061+0000 7fa449d89700 1 -- 172.21.15.96:0/3412658024 shutdown_connections 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.061+0000 7fa449d89700 1 -- 172.21.15.96:0/3412658024 wait complete. 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.061+0000 7fa449d89700 1 Processor -- start 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.061+0000 7fa449d89700 1 -- start start 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.062+0000 7fa449d89700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:57.983 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.062+0000 7fa449d89700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410df20 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.062+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.062+0000 7fa4437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33054/0 (socket says 172.21.15.96:33054) 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.062+0000 7fa4437fe700 1 -- 172.21.15.96:0/3074993869 learned_addr learned my addr 172.21.15.96:0/3074993869 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.062+0000 7fa4437fe700 1 -- 172.21.15.96:0/3074993869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44419e260 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.062+0000 7fa4437fe700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44419dd20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa4340119f0 tx=0x7fa434011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.063+0000 7fa4417fa700 1 -- 172.21.15.96:0/3074993869 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4340039e0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.063+0000 7fa4417fa700 1 -- 172.21.15.96:0/3074993869 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa43401b8f0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.063+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44419e4f0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.063+0000 7fa4417fa700 1 -- 172.21.15.96:0/3074993869 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4340237a0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.063+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4441a10f0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.064+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa444064cf0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.064+0000 7fa4417fa700 1 -- 172.21.15.96:0/3074993869 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fa434023aa0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.064+0000 7fa4417fa700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 0x7fa43003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.064+0000 7fa4417fa700 1 -- 172.21.15.96:0/3074993869 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa434052cd0 con 0x7fa44410af80 2024-04-25T17:00:57.984 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.064+0000 7fa43affd700 1 -- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 msgr2=0x7fa43003fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1771523969 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.064+0000 7fa43affd700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 0x7fa43003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.070+0000 7fa4417fa700 1 -- 172.21.15.96:0/3074993869 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa43401f030 con 0x7fa44410af80 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.265+0000 7fa43affd700 1 -- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 msgr2=0x7fa43003fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1771523969 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.265+0000 7fa43affd700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 0x7fa43003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.365+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa4441a13e0 con 0x7fa44410af80 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.366+0000 7fa4417fa700 1 -- 172.21.15.96:0/3074993869 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa434019e20 con 0x7fa44410af80 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 msgr2=0x7fa43003fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 0x7fa43003fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 msgr2=0x7fa44419dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44419dd20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa4340119f0 tx=0x7fa434011ad0 comp rx=0 tx=0).stop 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 shutdown_connections 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7fa43003d570 0x7fa43003fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 --2- 172.21.15.96:0/3074993869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa44410af80 0x7fa44419dd20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 >> 172.21.15.96:0/3074993869 conn(0x7fa4441036c0 msgr2=0x7fa444105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 shutdown_connections 2024-04-25T17:00:57.985 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:57.370+0000 7fa449d89700 1 -- 172.21.15.96:0/3074993869 wait complete. 2024-04-25T17:00:57.986 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-04-25T17:00:57.986 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 9... 2024-04-25T17:00:58.081 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:00:57 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3074993869' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T17:00:58.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:58.146+0000 7ff8f10a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:00:58.359 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:58.329+0000 7ff8f10a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:00:58.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:58.505+0000 7ff8f10a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:00:59.093 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:58.799+0000 7ff8f10a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:00:59.093 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:58.893+0000 7ff8f10a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:00:59.093 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:59.095+0000 7ff8f10a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:00:59.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:59.291+0000 7ff8f10a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:01:00.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:00:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:00:59.893+0000 7ff8f10a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:01:00.618 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:00.321+0000 7ff8f10a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:01:00.619 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:00.419+0000 7ff8f10a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:01:00.619 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:00.621+0000 7ff8f10a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:01:01.409 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:01.144+0000 7ff8f10a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:01:01.409 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:01.239+0000 7ff8f10a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:01:01.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:01.487+0000 7ff8f10a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:01:01.687 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:01.624+0000 7ff8f10a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:01:02.135 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:01.723+0000 7ff8f10a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:01:02.135 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:01.839+0000 7ff8f10a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:01:02.135 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:02.137+0000 7ff8f10a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:01:03.007 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:02.736+0000 7ff8f10a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:01:03.007 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:01:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:01:02.954+0000 7ff8f10a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:01:03.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: Active manager daemon x restarted 2024-04-25T17:01:03.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: Activating manager daemon x 2024-04-25T17:01:03.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: mgrmap e10: x(active, starting, since 0.00680455s) 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: Manager daemon x is now available 2024-04-25T17:01:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T17:01:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T17:01:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:04 smithi096 ceph-mon[29131]: mgrmap e11: x(active, since 1.00705s) 2024-04-25T17:01:04.885 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T17:01:04.885 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T17:01:04.885 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T17:01:04.885 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86ec860700 1 Processor -- start 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86ec860700 1 -- start start 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86ec860700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e8108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86ec860700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8108630 con 0x7f86e8107c50 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86e659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e8108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86e659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e8108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49816/0 (socket says 172.21.15.96:49816) 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86e659c700 1 -- 172.21.15.96:0/2914501634 learned_addr learned my addr 172.21.15.96:0/2914501634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.402+0000 7f86e659c700 1 -- 172.21.15.96:0/2914501634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e810ac50 con 0x7f86e8107c50 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.403+0000 7f86e659c700 1 --2- 172.21.15.96:0/2914501634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e8108060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0098a0 tx=0x7f86dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf478d1787f8701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.403+0000 7f86e559a700 1 -- 172.21.15.96:0/2914501634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc004160 con 0x7f86e8107c50 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.403+0000 7f86e559a700 1 -- 172.21.15.96:0/2914501634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86dc004300 con 0x7f86e8107c50 2024-04-25T17:01:04.886 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.403+0000 7f86e559a700 1 -- 172.21.15.96:0/2914501634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc011a20 con 0x7f86e8107c50 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.404+0000 7f86ec860700 1 -- 172.21.15.96:0/2914501634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 msgr2=0x7f86e8108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.404+0000 7f86ec860700 1 --2- 172.21.15.96:0/2914501634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e8108060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0098a0 tx=0x7f86dc009bb0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.404+0000 7f86ec860700 1 -- 172.21.15.96:0/2914501634 shutdown_connections 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.404+0000 7f86ec860700 1 --2- 172.21.15.96:0/2914501634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e8108060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.404+0000 7f86ec860700 1 -- 172.21.15.96:0/2914501634 >> 172.21.15.96:0/2914501634 conn(0x7f86e81036c0 msgr2=0x7f86e8105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.404+0000 7f86ec860700 1 -- 172.21.15.96:0/2914501634 shutdown_connections 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.404+0000 7f86ec860700 1 -- 172.21.15.96:0/2914501634 wait complete. 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.405+0000 7f86ec860700 1 Processor -- start 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.405+0000 7f86ec860700 1 -- start start 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.405+0000 7f86ec860700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.405+0000 7f86ec860700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86e810de80 con 0x7f86e8107c50 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.405+0000 7f86e659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.405+0000 7f86e659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49828/0 (socket says 172.21.15.96:49828) 2024-04-25T17:01:04.887 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.405+0000 7f86e659c700 1 -- 172.21.15.96:0/373875875 learned_addr learned my addr 172.21.15.96:0/373875875 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.406+0000 7f86e659c700 1 -- 172.21.15.96:0/373875875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e819e420 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.406+0000 7f86e659c700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e819dee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86dc003f00 tx=0x7f86dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.406+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc003aa0 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.406+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e819e6b0 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.406+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86dc01b8f0 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.406+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86dc023890 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.406+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e81a12c0 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.408+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f86dc02c3f0 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.408+0000 7f86d77fe700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.408+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 --> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86d00401e0 con 0x7f86d003d5c0 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.408+0000 7f86e5d9b700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 msgr2=0x7f86d003fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1771523969 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.408+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f86dc052da0 con 0x7f86e8107c50 2024-04-25T17:01:04.888 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.408+0000 7f86e5d9b700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.609+0000 7f86e5d9b700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 msgr2=0x7f86d003fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1771523969 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:58.609+0000 7f86e5d9b700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:59.010+0000 7f86e5d9b700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 msgr2=0x7f86d003fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1771523969 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:59.010+0000 7f86e5d9b700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:59.811+0000 7f86e5d9b700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 msgr2=0x7f86d003fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1771523969 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:00:59.811+0000 7f86e5d9b700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:01.412+0000 7f86e5d9b700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 msgr2=0x7f86d003fad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1771523969 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:01.412+0000 7f86e5d9b700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:02.961+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7f86dc02cb00 con 0x7f86e8107c50 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:02.961+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 msgr2=0x7f86d003fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:02.961+0000 7f86d77fe700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.960+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f86dc054920 con 0x7f86e8107c50 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.960+0000 7f86d77fe700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f86dc052930 0x7f86d0043000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:04.889 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.960+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86dc009530 con 0x7f86dc052930 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.961+0000 7f86e5d9b700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f86dc052930 0x7f86d0043000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.961+0000 7f86e5d9b700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f86dc052930 0x7f86d0043000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f86d8003a10 tx=0x7f86d80092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.962+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f86dc009530 con 0x7f86dc052930 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.970+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f86e8064cf0 con 0x7f86dc052930 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86d77fe700 1 -- 172.21.15.96:0/373875875 <== mgr.14148 v2:172.21.15.96:6800/3530143863 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f86e8064cf0 con 0x7f86dc052930 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f86dc052930 msgr2=0x7f86d0043000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f86dc052930 0x7f86d0043000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f86d8003a10 tx=0x7f86d80092b0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 msgr2=0x7f86e819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e819dee0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86dc003f00 tx=0x7f86dc011aa0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 shutdown_connections 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f86dc052930 0x7f86d0043000 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86e8107c50 0x7f86e819dee0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 --2- 172.21.15.96:0/373875875 >> [v2:172.21.15.96:6800/1771523969,v1:172.21.15.96:6801/1771523969] conn(0x7f86d003d5c0 0x7f86d003fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:04.890 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 >> 172.21.15.96:0/373875875 conn(0x7f86e81036c0 msgr2=0x7f86e8105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:04.891 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 shutdown_connections 2024-04-25T17:01:04.891 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:03.971+0000 7f86ec860700 1 -- 172.21.15.96:0/373875875 wait complete. 2024-04-25T17:01:04.891 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 9 is available 2024-04-25T17:01:04.891 INFO:teuthology.orchestra.run.smithi096.stdout:Generating a dashboard self-signed certificate... 2024-04-25T17:01:05.130 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:05 smithi096 ceph-mon[29131]: [25/Apr/2024:17:01:03] ENGINE Bus STARTING 2024-04-25T17:01:05.130 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:05 smithi096 ceph-mon[29131]: [25/Apr/2024:17:01:03] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T17:01:05.131 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:05 smithi096 ceph-mon[29131]: [25/Apr/2024:17:01:03] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T17:01:05.131 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:05 smithi096 ceph-mon[29131]: [25/Apr/2024:17:01:03] ENGINE Bus STARTED 2024-04-25T17:01:05.131 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:05 smithi096 ceph-mon[29131]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T17:01:05.131 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:05 smithi096 ceph-mon[29131]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T17:01:05.131 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:06.232 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.347+0000 7f2d2c544700 1 Processor -- start 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.347+0000 7f2d2c544700 1 -- start start 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d2c544700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d24108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d2c544700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241092a0 con 0x7f2d241088a0 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d2a2e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d24108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d2a2e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d24108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49886/0 (socket says 172.21.15.96:49886) 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d2a2e0700 1 -- 172.21.15.96:0/3197921637 learned_addr learned my addr 172.21.15.96:0/3197921637 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d2a2e0700 1 -- 172.21.15.96:0/3197921637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d24109a70 con 0x7f2d241088a0 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d2a2e0700 1 --2- 172.21.15.96:0/3197921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d24108cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d200098a0 tx=0x7f2d20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b89d8f7dfe4494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d292de700 1 -- 172.21.15.96:0/3197921637 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d20004160 con 0x7f2d241088a0 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d292de700 1 -- 172.21.15.96:0/3197921637 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d20004300 con 0x7f2d241088a0 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.348+0000 7f2d292de700 1 -- 172.21.15.96:0/3197921637 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d20011a20 con 0x7f2d241088a0 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.349+0000 7f2d2c544700 1 -- 172.21.15.96:0/3197921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 msgr2=0x7f2d24108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:06.233 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.349+0000 7f2d2c544700 1 --2- 172.21.15.96:0/3197921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d24108cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d200098a0 tx=0x7f2d20009bb0 comp rx=0 tx=0).stop 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.349+0000 7f2d2c544700 1 -- 172.21.15.96:0/3197921637 shutdown_connections 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.349+0000 7f2d2c544700 1 --2- 172.21.15.96:0/3197921637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d24108cb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.349+0000 7f2d2c544700 1 -- 172.21.15.96:0/3197921637 >> 172.21.15.96:0/3197921637 conn(0x7f2d24103f20 msgr2=0x7f2d24106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.349+0000 7f2d2c544700 1 -- 172.21.15.96:0/3197921637 shutdown_connections 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.349+0000 7f2d2c544700 1 -- 172.21.15.96:0/3197921637 wait complete. 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.350+0000 7f2d2c544700 1 Processor -- start 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.350+0000 7f2d2c544700 1 -- start start 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.350+0000 7f2d2c544700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d2419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.350+0000 7f2d2c544700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410ce20 con 0x7f2d241088a0 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.350+0000 7f2d2a2e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d2419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.350+0000 7f2d2a2e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d2419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49894/0 (socket says 172.21.15.96:49894) 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.350+0000 7f2d2a2e0700 1 -- 172.21.15.96:0/743504276 learned_addr learned my addr 172.21.15.96:0/743504276 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.351+0000 7f2d2a2e0700 1 -- 172.21.15.96:0/743504276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2419d240 con 0x7f2d241088a0 2024-04-25T17:01:06.234 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.351+0000 7f2d2a2e0700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d2419cd00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009130 tx=0x7f2d20011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.351+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d20003aa0 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.351+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2419d4d0 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.351+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2001a8e0 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.351+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d200248f0 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.351+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a00e0 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.352+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f2d20022020 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.352+0000 7f2d1b7fe700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2d1003d4a0 0x7f2d1003f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.352+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d24064cf0 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.352+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2d20053520 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.353+0000 7f2d29adf700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2d1003d4a0 0x7f2d1003f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.353+0000 7f2d29adf700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2d1003d4a0 0x7f2d1003f9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d14006fd0 tx=0x7f2d14006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.358+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d20019c70 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.519+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f2d2004cd10 con 0x7f2d241088a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.571+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f2d24104e90 con 0x7f2d1003d4a0 2024-04-25T17:01:06.235 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.671+0000 7f2d1b7fe700 1 -- 172.21.15.96:0/743504276 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f2d24104e90 con 0x7f2d1003d4a0 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2d1003d4a0 msgr2=0x7f2d1003f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2d1003d4a0 0x7f2d1003f9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d14006fd0 tx=0x7f2d14006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 msgr2=0x7f2d2419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d2419cd00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009130 tx=0x7f2d20011aa0 comp rx=0 tx=0).stop 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 shutdown_connections 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2d1003d4a0 0x7f2d1003f9b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 --2- 172.21.15.96:0/743504276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d241088a0 0x7f2d2419cd00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 >> 172.21.15.96:0/743504276 conn(0x7f2d24103f20 msgr2=0x7f2d24104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 shutdown_connections 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:05.673+0000 7f2d2c544700 1 -- 172.21.15.96:0/743504276 wait complete. 2024-04-25T17:01:06.236 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial admin user... 2024-04-25T17:01:06.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:06 smithi096 ceph-mon[29131]: mgrmap e12: x(active, since 2s) 2024-04-25T17:01:06.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:06 smithi096 ceph-mon[29131]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:06.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:06.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:06.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:07.790 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:07 smithi096 ceph-mon[29131]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:07.791 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:08.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$oJ451wsQ5Haidj7fLADuGeAZMTaEzfFfIPHX8Eh0tbwUESMfhjvK2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714064467, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T17:01:08.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.680+0000 7f248d91d700 1 Processor -- start 2024-04-25T17:01:08.419 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.680+0000 7f248d91d700 1 -- start start 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.680+0000 7f248d91d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248810c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.680+0000 7f248d91d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f248807d6b0 con 0x7f248810a4e0 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.680+0000 7f2486ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248810c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.680+0000 7f2486ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248810c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49900/0 (socket says 172.21.15.96:49900) 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.680+0000 7f2486ffd700 1 -- 172.21.15.96:0/1814131702 learned_addr learned my addr 172.21.15.96:0/1814131702 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.681+0000 7f2486ffd700 1 -- 172.21.15.96:0/1814131702 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248810ce40 con 0x7f248810a4e0 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.681+0000 7f2486ffd700 1 --2- 172.21.15.96:0/1814131702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248810c900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24780098a0 tx=0x7f2478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc316df4f1b5810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.681+0000 7f2485ffb700 1 -- 172.21.15.96:0/1814131702 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2478004160 con 0x7f248810a4e0 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.681+0000 7f2485ffb700 1 -- 172.21.15.96:0/1814131702 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2478004300 con 0x7f248810a4e0 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.681+0000 7f2485ffb700 1 -- 172.21.15.96:0/1814131702 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2478011a20 con 0x7f248810a4e0 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 -- 172.21.15.96:0/1814131702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 msgr2=0x7f248810c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 --2- 172.21.15.96:0/1814131702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248810c900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24780098a0 tx=0x7f2478009bb0 comp rx=0 tx=0).stop 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 -- 172.21.15.96:0/1814131702 shutdown_connections 2024-04-25T17:01:08.420 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 --2- 172.21.15.96:0/1814131702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248810c900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 -- 172.21.15.96:0/1814131702 >> 172.21.15.96:0/1814131702 conn(0x7f2488103f20 msgr2=0x7f2488106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 -- 172.21.15.96:0/1814131702 shutdown_connections 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 -- 172.21.15.96:0/1814131702 wait complete. 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.682+0000 7f248d91d700 1 Processor -- start 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f248d91d700 1 -- start start 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f248d91d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f248d91d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f248810dde0 con 0x7f248810a4e0 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f2486ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f2486ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49908/0 (socket says 172.21.15.96:49908) 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f2486ffd700 1 -- 172.21.15.96:0/860829163 learned_addr learned my addr 172.21.15.96:0/860829163 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f2486ffd700 1 -- 172.21.15.96:0/860829163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248819e2e0 con 0x7f248810a4e0 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.683+0000 7f2486ffd700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248819dda0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2478004660 tx=0x7f2478011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.684+0000 7f248c91b700 1 -- 172.21.15.96:0/860829163 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2478003a60 con 0x7f248810a4e0 2024-04-25T17:01:08.421 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.684+0000 7f248c91b700 1 -- 172.21.15.96:0/860829163 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247801b8f0 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.684+0000 7f248c91b700 1 -- 172.21.15.96:0/860829163 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2478023890 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.684+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248819e570 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.684+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24881a1180 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.685+0000 7f248c91b700 1 -- 172.21.15.96:0/860829163 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f247802c3f0 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.685+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2488064cf0 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.685+0000 7f248c91b700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f247003d570 0x7f247003fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.685+0000 7f248c91b700 1 -- 172.21.15.96:0/860829163 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2478052ee0 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.685+0000 7f24867fc700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f247003d570 0x7f247003fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.686+0000 7f24867fc700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f247003d570 0x7f247003fa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f247c006fd0 tx=0x7f247c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.690+0000 7f248c91b700 1 -- 172.21.15.96:0/860829163 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f247801f030 con 0x7f248810a4e0 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:06.924+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f24881a1410 con 0x7f247003d570 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.149+0000 7f248c91b700 1 -- 172.21.15.96:0/860829163 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f24881a1410 con 0x7f247003d570 2024-04-25T17:01:08.422 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.151+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f247003d570 msgr2=0x7f247003fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.151+0000 7f248d91d700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f247003d570 0x7f247003fa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f247c006fd0 tx=0x7f247c006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.151+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 msgr2=0x7f248819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.151+0000 7f248d91d700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248819dda0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2478004660 tx=0x7f2478011aa0 comp rx=0 tx=0).stop 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.152+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 shutdown_connections 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.152+0000 7f248d91d700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f247003d570 0x7f247003fa80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.152+0000 7f248d91d700 1 --2- 172.21.15.96:0/860829163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f248810a4e0 0x7f248819dda0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.152+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 >> 172.21.15.96:0/860829163 conn(0x7f2488103f20 msgr2=0x7f24881047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.152+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 shutdown_connections 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:07.152+0000 7f248d91d700 1 -- 172.21.15.96:0/860829163 wait complete. 2024-04-25T17:01:08.423 INFO:teuthology.orchestra.run.smithi096.stdout:Fetching dashboard port number... 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.845+0000 7fb1ffbbe700 1 Processor -- start 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.845+0000 7fb1ffbbe700 1 -- start start 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.846+0000 7fb1ffbbe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f810c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.846+0000 7fb1ffbbe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f807d870 con 0x7fb1f810a460 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.846+0000 7fb1fd95a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f810c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.846+0000 7fb1fd95a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f810c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48154/0 (socket says 172.21.15.96:48154) 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.846+0000 7fb1fd95a700 1 -- 172.21.15.96:0/915665823 learned_addr learned my addr 172.21.15.96:0/915665823 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:09.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.846+0000 7fb1fd95a700 1 -- 172.21.15.96:0/915665823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f810cdc0 con 0x7fb1f810a460 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.846+0000 7fb1fd95a700 1 --2- 172.21.15.96:0/915665823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f810c880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40098a0 tx=0x7fb1f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c6a51ac7eaff5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1fc958700 1 -- 172.21.15.96:0/915665823 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f4004160 con 0x7fb1f810a460 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1fc958700 1 -- 172.21.15.96:0/915665823 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f4004300 con 0x7fb1f810a460 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1fc958700 1 -- 172.21.15.96:0/915665823 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f4011a20 con 0x7fb1f810a460 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/915665823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 msgr2=0x7fb1f810c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1ffbbe700 1 --2- 172.21.15.96:0/915665823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f810c880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40098a0 tx=0x7fb1f4009bb0 comp rx=0 tx=0).stop 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/915665823 shutdown_connections 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1ffbbe700 1 --2- 172.21.15.96:0/915665823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f810c880 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/915665823 >> 172.21.15.96:0/915665823 conn(0x7fb1f8103e80 msgr2=0x7fb1f8106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.847+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/915665823 shutdown_connections 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/915665823 wait complete. 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1ffbbe700 1 Processor -- start 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1ffbbe700 1 -- start start 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1ffbbe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:09.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1ffbbe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f810dd60 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1fd95a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1fd95a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48160/0 (socket says 172.21.15.96:48160) 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.848+0000 7fb1fd95a700 1 -- 172.21.15.96:0/977727721 learned_addr learned my addr 172.21.15.96:0/977727721 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.849+0000 7fb1fd95a700 1 -- 172.21.15.96:0/977727721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f819e280 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.849+0000 7fb1fd95a700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f819dd40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009520 tx=0x7fb1f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.849+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f40039e0 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.849+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f401b8f0 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.849+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f819e510 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.849+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f40197d0 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.849+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f81a1110 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.850+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f8064cf0 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.850+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7fb1f4019a40 con 0x7fb1f810a460 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.850+0000 7fb1eeffd700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb1e403d620 0x7fb1e403fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.850+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb1f4052e90 con 0x7fb1f810a460 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.850+0000 7fb1fd159700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb1e403d620 0x7fb1e403fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.851+0000 7fb1fd159700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb1e403d620 0x7fb1e403fb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8006fd0 tx=0x7fb1e8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:08.855+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1f401f030 con 0x7fb1f810a460 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.009+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fb1f401a3d0 con 0x7fb1f810a460 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.057+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb1f81a12e0 con 0x7fb1f810a460 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.057+0000 7fb1eeffd700 1 -- 172.21.15.96:0/977727721 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb1f402a570 con 0x7fb1f810a460 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb1e403d620 msgr2=0x7fb1e403fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb1e403d620 0x7fb1e403fb30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8006fd0 tx=0x7fb1e8006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 msgr2=0x7fb1f819dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f819dd40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009520 tx=0x7fb1f4011aa0 comp rx=0 tx=0).stop 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 shutdown_connections 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb1e403d620 0x7fb1e403fb30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 --2- 172.21.15.96:0/977727721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1f810a460 0x7fb1f819dd40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:09.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.059+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 >> 172.21.15.96:0/977727721 conn(0x7fb1f8103e80 msgr2=0x7fb1f8104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:09.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.060+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 shutdown_connections 2024-04-25T17:01:09.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:09.060+0000 7fb1ffbbe700 1 -- 172.21.15.96:0/977727721 wait complete. 2024-04-25T17:01:09.623 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T17:01:09.623 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T17:01:09.630 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T17:01:09.630 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T17:01:09.630 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T17:01:09.630 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T17:01:09.632 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph Dashboard is now available at: 2024-04-25T17:01:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: URL: https://smithi096:8443/ 2024-04-25T17:01:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: User: admin 2024-04-25T17:01:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: Password: 57kgrs5uvs 2024-04-25T17:01:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:09.633 INFO:teuthology.orchestra.run.smithi096.stdout:Saving cluster configuration to /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config directory 2024-04-25T17:01:09.633 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling autotune for osd_memory_target 2024-04-25T17:01:10.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:10 smithi096 ceph-mon[29131]: mgrmap e13: x(active, since 6s) 2024-04-25T17:01:10.266 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:10 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/977727721' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T17:01:10.826 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.059+0000 7fdf78abf700 1 Processor -- start 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.060+0000 7fdf78abf700 1 -- start start 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.060+0000 7fdf78abf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7410c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.060+0000 7fdf78abf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7407d6d0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.060+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7410c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.060+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7410c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48170/0 (socket says 172.21.15.96:48170) 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.060+0000 7fdf7259c700 1 -- 172.21.15.96:0/1628779847 learned_addr learned my addr 172.21.15.96:0/1628779847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.060+0000 7fdf7259c700 1 -- 172.21.15.96:0/1628779847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7410ce20 con 0x7fdf7410a4c0 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.061+0000 7fdf7259c700 1 --2- 172.21.15.96:0/1628779847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7410c8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098a0 tx=0x7fdf64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdcfd60238b0439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.061+0000 7fdf7159a700 1 -- 172.21.15.96:0/1628779847 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf64004160 con 0x7fdf7410a4c0 2024-04-25T17:01:10.832 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.061+0000 7fdf7159a700 1 -- 172.21.15.96:0/1628779847 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf64004300 con 0x7fdf7410a4c0 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.061+0000 7fdf7159a700 1 -- 172.21.15.96:0/1628779847 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf64011a20 con 0x7fdf7410a4c0 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 -- 172.21.15.96:0/1628779847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 msgr2=0x7fdf7410c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 --2- 172.21.15.96:0/1628779847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7410c8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf640098a0 tx=0x7fdf64009bb0 comp rx=0 tx=0).stop 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 -- 172.21.15.96:0/1628779847 shutdown_connections 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 --2- 172.21.15.96:0/1628779847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7410c8e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 -- 172.21.15.96:0/1628779847 >> 172.21.15.96:0/1628779847 conn(0x7fdf74103f20 msgr2=0x7fdf74106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 -- 172.21.15.96:0/1628779847 shutdown_connections 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 -- 172.21.15.96:0/1628779847 wait complete. 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.062+0000 7fdf78abf700 1 Processor -- start 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf78abf700 1 -- start start 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf78abf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf78abf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410ddc0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.833 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf7259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7419dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48184/0 (socket says 172.21.15.96:48184) 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf7259c700 1 -- 172.21.15.96:0/1071030442 learned_addr learned my addr 172.21.15.96:0/1071030442 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf7259c700 1 -- 172.21.15.96:0/1071030442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7419e2c0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf7259c700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7419dd80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009520 tx=0x7fdf64011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf637fe700 1 -- 172.21.15.96:0/1071030442 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf640039e0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf637fe700 1 -- 172.21.15.96:0/1071030442 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf6401b8f0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7419e550 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.063+0000 7fdf637fe700 1 -- 172.21.15.96:0/1071030442 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf640197d0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.064+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a1160 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.064+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf74064cf0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.065+0000 7fdf637fe700 1 -- 172.21.15.96:0/1071030442 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fdf64019ab0 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.065+0000 7fdf637fe700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf5c03d660 0x7fdf5c03fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.065+0000 7fdf637fe700 1 -- 172.21.15.96:0/1071030442 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf64052f90 con 0x7fdf7410a4c0 2024-04-25T17:01:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.065+0000 7fdf71d9b700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf5c03d660 0x7fdf5c03fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.065+0000 7fdf71d9b700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf5c03d660 0x7fdf5c03fb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf68006fd0 tx=0x7fdf68006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.070+0000 7fdf637fe700 1 -- 172.21.15.96:0/1071030442 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf6401f030 con 0x7fdf7410a4c0 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.268+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fdf741a1330 con 0x7fdf7410a4c0 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.268+0000 7fdf637fe700 1 -- 172.21.15.96:0/1071030442 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fdf6402a350 con 0x7fdf7410a4c0 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf5c03d660 msgr2=0x7fdf5c03fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf5c03d660 0x7fdf5c03fb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdf68006fd0 tx=0x7fdf68006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 msgr2=0x7fdf7419dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7419dd80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009520 tx=0x7fdf64011aa0 comp rx=0 tx=0).stop 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 shutdown_connections 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf5c03d660 0x7fdf5c03fb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 --2- 172.21.15.96:0/1071030442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf7410a4c0 0x7fdf7419dd80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 >> 172.21.15.96:0/1071030442 conn(0x7fdf74103f20 msgr2=0x7fdf74104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 shutdown_connections 2024-04-25T17:01:10.835 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:10.270+0000 7fdf78abf700 1 -- 172.21.15.96:0/1071030442 wait complete. 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.271+0000 7f43c8a63700 1 Processor -- start 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.271+0000 7f43c8a63700 1 -- start start 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.271+0000 7f43c8a63700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c4108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.271+0000 7f43c8a63700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41092a0 con 0x7f43c41088a0 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c4108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c4108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48190/0 (socket says 172.21.15.96:48190) 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c259c700 1 -- 172.21.15.96:0/632751907 learned_addr learned my addr 172.21.15.96:0/632751907 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c259c700 1 -- 172.21.15.96:0/632751907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c4109a70 con 0x7f43c41088a0 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c259c700 1 --2- 172.21.15.96:0/632751907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c4108cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009e80 tx=0x7f43b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88cc16cdfae8198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:12.146 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c159a700 1 -- 172.21.15.96:0/632751907 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4017070 con 0x7f43c41088a0 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c159a700 1 -- 172.21.15.96:0/632751907 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b400b8c0 con 0x7f43c41088a0 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.272+0000 7f43c159a700 1 -- 172.21.15.96:0/632751907 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4004490 con 0x7f43c41088a0 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.273+0000 7f43c8a63700 1 -- 172.21.15.96:0/632751907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 msgr2=0x7f43c4108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.273+0000 7f43c8a63700 1 --2- 172.21.15.96:0/632751907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c4108cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009e80 tx=0x7f43b400b330 comp rx=0 tx=0).stop 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.273+0000 7f43c8a63700 1 -- 172.21.15.96:0/632751907 shutdown_connections 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.273+0000 7f43c8a63700 1 --2- 172.21.15.96:0/632751907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c4108cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.273+0000 7f43c8a63700 1 -- 172.21.15.96:0/632751907 >> 172.21.15.96:0/632751907 conn(0x7f43c4103f40 msgr2=0x7f43c4106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.273+0000 7f43c8a63700 1 -- 172.21.15.96:0/632751907 shutdown_connections 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.273+0000 7f43c8a63700 1 -- 172.21.15.96:0/632751907 wait complete. 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.274+0000 7f43c8a63700 1 Processor -- start 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.274+0000 7f43c8a63700 1 -- start start 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.274+0000 7f43c8a63700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.274+0000 7f43c8a63700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43c410ce20 con 0x7f43c41088a0 2024-04-25T17:01:12.147 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.274+0000 7f43c259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.274+0000 7f43c259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48198/0 (socket says 172.21.15.96:48198) 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.274+0000 7f43c259c700 1 -- 172.21.15.96:0/52881290 learned_addr learned my addr 172.21.15.96:0/52881290 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.275+0000 7f43c259c700 1 -- 172.21.15.96:0/52881290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c419d290 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.275+0000 7f43c259c700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c419cd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009e50 tx=0x7f43b400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.275+0000 7f43bb7fe700 1 -- 172.21.15.96:0/52881290 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4020020 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.275+0000 7f43bb7fe700 1 -- 172.21.15.96:0/52881290 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b4012430 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.275+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c419d520 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.275+0000 7f43bb7fe700 1 -- 172.21.15.96:0/52881290 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4017040 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.275+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c41a0120 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.276+0000 7f43bb7fe700 1 -- 172.21.15.96:0/52881290 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f43b4023020 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.276+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c4064cf0 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.276+0000 7f43bb7fe700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43b003d670 0x7f43b003fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.276+0000 7f43bb7fe700 1 -- 172.21.15.96:0/52881290 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f43b400f050 con 0x7f43c41088a0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.277+0000 7f43c1d9b700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43b003d670 0x7f43b003fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.277+0000 7f43c1d9b700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43b003d670 0x7f43b003fb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f43ac006fd0 tx=0x7f43ac006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.282+0000 7f43bb7fe700 1 -- 172.21.15.96:0/52881290 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43b402a020 con 0x7f43c41088a0 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.589+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f43c41a02f0 con 0x7f43c41088a0 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.592+0000 7f43bb7fe700 1 -- 172.21.15.96:0/52881290 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f43b4011830 con 0x7f43c41088a0 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43b003d670 msgr2=0x7f43b003fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43b003d670 0x7f43b003fb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f43ac006fd0 tx=0x7f43ac006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 msgr2=0x7f43c419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c419cd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009e50 tx=0x7f43b400bd50 comp rx=0 tx=0).stop 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 shutdown_connections 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43b003d670 0x7f43b003fb80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 --2- 172.21.15.96:0/52881290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43c41088a0 0x7f43c419cd50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 >> 172.21.15.96:0/52881290 conn(0x7f43c4103f40 msgr2=0x7f43c41047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 shutdown_connections 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T17:01:11.594+0000 7f43c8a63700 1 -- 172.21.15.96:0/52881290 wait complete. 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T17:01:12.149 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: ceph telemetry on 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout:For more information see: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:12.150 INFO:teuthology.orchestra.run.smithi096.stdout:Bootstrap complete. 2024-04-25T17:01:12.181 INFO:tasks.cephadm:Fetching config... 2024-04-25T17:01:12.182 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:01:12.182 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T17:01:12.197 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T17:01:12.197 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:01:12.197 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T17:01:12.253 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T17:01:12.253 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:01:12.253 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T17:01:12.317 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T17:01:12.317 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:01:12.317 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T17:01:12.373 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T17:01:12.373 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0MrRAlSpiTTpC0JSNXwgHqO5EaQtn/79plIXYVqg7XcK/mTyLnh+8tDjhrGomj3LkeUGYVUq6PzTyfNPq2Ny9/OonpfAgYDwU07df6H0NnjWatHn4Jom2eSODf5eQ6rY1z7+7+B9z+n9aJFMT+4CkwQ9VNmtOTouUP5Pl3+yWCXxe6FtMRbpDfxXYy+lsaAVROxS0NLpzoaGEMd51vkQ1J5oS0fdPdodKL34gLuc1Gu8PtPKmRxYNO97uz2Njs1CTILOqGnC9WcwVrCOi7z/IQNGhGMvZYBm5PrmRad705WBNUCQEzNYRct0ZeFv50jCS65I9c0c11SRki9pEWwSDZKhbk05uUXRrph3sH0Oc2gSS99n5Hi0YeJuy1jlx4sI8pZzjdFZwLcoB++V32XZIRVNQ7IFflu0C2xxe9L1FSJbxtdLCv4p9bgiXualJ6f/Ls4y1kKvlQO03pfK80YUPa4G0ia1XD5aJEvIpkUZhpcMXr6G3M0cU2JU8UQZUXfM= ceph-1be09d02-0325-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T17:01:12.449 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0MrRAlSpiTTpC0JSNXwgHqO5EaQtn/79plIXYVqg7XcK/mTyLnh+8tDjhrGomj3LkeUGYVUq6PzTyfNPq2Ny9/OonpfAgYDwU07df6H0NnjWatHn4Jom2eSODf5eQ6rY1z7+7+B9z+n9aJFMT+4CkwQ9VNmtOTouUP5Pl3+yWCXxe6FtMRbpDfxXYy+lsaAVROxS0NLpzoaGEMd51vkQ1J5oS0fdPdodKL34gLuc1Gu8PtPKmRxYNO97uz2Njs1CTILOqGnC9WcwVrCOi7z/IQNGhGMvZYBm5PrmRad705WBNUCQEzNYRct0ZeFv50jCS65I9c0c11SRki9pEWwSDZKhbk05uUXRrph3sH0Oc2gSS99n5Hi0YeJuy1jlx4sI8pZzjdFZwLcoB++V32XZIRVNQ7IFflu0C2xxe9L1FSJbxtdLCv4p9bgiXualJ6f/Ls4y1kKvlQO03pfK80YUPa4G0ia1XD5aJEvIpkUZhpcMXr6G3M0cU2JU8UQZUXfM= ceph-1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:01:12.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0MrRAlSpiTTpC0JSNXwgHqO5EaQtn/79plIXYVqg7XcK/mTyLnh+8tDjhrGomj3LkeUGYVUq6PzTyfNPq2Ny9/OonpfAgYDwU07df6H0NnjWatHn4Jom2eSODf5eQ6rY1z7+7+B9z+n9aJFMT+4CkwQ9VNmtOTouUP5Pl3+yWCXxe6FtMRbpDfxXYy+lsaAVROxS0NLpzoaGEMd51vkQ1J5oS0fdPdodKL34gLuc1Gu8PtPKmRxYNO97uz2Njs1CTILOqGnC9WcwVrCOi7z/IQNGhGMvZYBm5PrmRad705WBNUCQEzNYRct0ZeFv50jCS65I9c0c11SRki9pEWwSDZKhbk05uUXRrph3sH0Oc2gSS99n5Hi0YeJuy1jlx4sI8pZzjdFZwLcoB++V32XZIRVNQ7IFflu0C2xxe9L1FSJbxtdLCv4p9bgiXualJ6f/Ls4y1kKvlQO03pfK80YUPa4G0ia1XD5aJEvIpkUZhpcMXr6G3M0cU2JU8UQZUXfM= ceph-1be09d02-0325-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T17:01:12.505 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0MrRAlSpiTTpC0JSNXwgHqO5EaQtn/79plIXYVqg7XcK/mTyLnh+8tDjhrGomj3LkeUGYVUq6PzTyfNPq2Ny9/OonpfAgYDwU07df6H0NnjWatHn4Jom2eSODf5eQ6rY1z7+7+B9z+n9aJFMT+4CkwQ9VNmtOTouUP5Pl3+yWCXxe6FtMRbpDfxXYy+lsaAVROxS0NLpzoaGEMd51vkQ1J5oS0fdPdodKL34gLuc1Gu8PtPKmRxYNO97uz2Njs1CTILOqGnC9WcwVrCOi7z/IQNGhGMvZYBm5PrmRad705WBNUCQEzNYRct0ZeFv50jCS65I9c0c11SRki9pEWwSDZKhbk05uUXRrph3sH0Oc2gSS99n5Hi0YeJuy1jlx4sI8pZzjdFZwLcoB++V32XZIRVNQ7IFflu0C2xxe9L1FSJbxtdLCv4p9bgiXualJ6f/Ls4y1kKvlQO03pfK80YUPa4G0ia1XD5aJEvIpkUZhpcMXr6G3M0cU2JU8UQZUXfM= ceph-1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:01:12.517 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0MrRAlSpiTTpC0JSNXwgHqO5EaQtn/79plIXYVqg7XcK/mTyLnh+8tDjhrGomj3LkeUGYVUq6PzTyfNPq2Ny9/OonpfAgYDwU07df6H0NnjWatHn4Jom2eSODf5eQ6rY1z7+7+B9z+n9aJFMT+4CkwQ9VNmtOTouUP5Pl3+yWCXxe6FtMRbpDfxXYy+lsaAVROxS0NLpzoaGEMd51vkQ1J5oS0fdPdodKL34gLuc1Gu8PtPKmRxYNO97uz2Njs1CTILOqGnC9WcwVrCOi7z/IQNGhGMvZYBm5PrmRad705WBNUCQEzNYRct0ZeFv50jCS65I9c0c11SRki9pEWwSDZKhbk05uUXRrph3sH0Oc2gSS99n5Hi0YeJuy1jlx4sI8pZzjdFZwLcoB++V32XZIRVNQ7IFflu0C2xxe9L1FSJbxtdLCv4p9bgiXualJ6f/Ls4y1kKvlQO03pfK80YUPa4G0ia1XD5aJEvIpkUZhpcMXr6G3M0cU2JU8UQZUXfM= ceph-1be09d02-0325-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T17:01:12.558 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0MrRAlSpiTTpC0JSNXwgHqO5EaQtn/79plIXYVqg7XcK/mTyLnh+8tDjhrGomj3LkeUGYVUq6PzTyfNPq2Ny9/OonpfAgYDwU07df6H0NnjWatHn4Jom2eSODf5eQ6rY1z7+7+B9z+n9aJFMT+4CkwQ9VNmtOTouUP5Pl3+yWCXxe6FtMRbpDfxXYy+lsaAVROxS0NLpzoaGEMd51vkQ1J5oS0fdPdodKL34gLuc1Gu8PtPKmRxYNO97uz2Njs1CTILOqGnC9WcwVrCOi7z/IQNGhGMvZYBm5PrmRad705WBNUCQEzNYRct0ZeFv50jCS65I9c0c11SRki9pEWwSDZKhbk05uUXRrph3sH0Oc2gSS99n5Hi0YeJuy1jlx4sI8pZzjdFZwLcoB++V32XZIRVNQ7IFflu0C2xxe9L1FSJbxtdLCv4p9bgiXualJ6f/Ls4y1kKvlQO03pfK80YUPa4G0ia1XD5aJEvIpkUZhpcMXr6G3M0cU2JU8UQZUXfM= ceph-1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:01:12.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T17:01:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:12 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/52881290' entity='client.admin' 2024-04-25T17:01:12.808 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 -- 172.21.15.96:0/2440341007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 msgr2=0x7fa328102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 --2- 172.21.15.96:0/2440341007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa328102db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa318009910 tx=0x7fa318009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 -- 172.21.15.96:0/2440341007 shutdown_connections 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 --2- 172.21.15.96:0/2440341007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa328102db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 -- 172.21.15.96:0/2440341007 >> 172.21.15.96:0/2440341007 conn(0x7fa3280fdfe0 msgr2=0x7fa328100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 -- 172.21.15.96:0/2440341007 shutdown_connections 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 -- 172.21.15.96:0/2440341007 wait complete. 2024-04-25T17:01:14.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 Processor -- start 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.997+0000 7fa32dccf700 1 -- start start 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa32dccf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa32806c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa32dccf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa328108960 con 0x7fa3281029a0 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa3277fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa32806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa3277fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa32806c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48214/0 (socket says 172.21.15.96:48214) 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa3277fe700 1 -- 172.21.15.96:0/3325009697 learned_addr learned my addr 172.21.15.96:0/3325009697 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa3277fe700 1 -- 172.21.15.96:0/3325009697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32806c5d0 con 0x7fa3281029a0 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa3277fe700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa32806c090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3180057d0 tx=0x7fa318005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa324ff9700 1 -- 172.21.15.96:0/3325009697 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa318027070 con 0x7fa3281029a0 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32819c350 con 0x7fa3281029a0 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa324ff9700 1 -- 172.21.15.96:0/3325009697 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31800be10 con 0x7fa3281029a0 2024-04-25T17:01:14.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa324ff9700 1 -- 172.21.15.96:0/3325009697 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa318022dd0 con 0x7fa3281029a0 2024-04-25T17:01:14.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.998+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32819c7d0 con 0x7fa3281029a0 2024-04-25T17:01:14.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.999+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3281090e0 con 0x7fa3281029a0 2024-04-25T17:01:15.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:14.999+0000 7fa324ff9700 1 -- 172.21.15.96:0/3325009697 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fa31802a070 con 0x7fa3281029a0 2024-04-25T17:01:15.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.000+0000 7fa324ff9700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa31003d5c0 0x7fa31003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:15.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.000+0000 7fa324ff9700 1 -- 172.21.15.96:0/3325009697 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa31801e030 con 0x7fa3281029a0 2024-04-25T17:01:15.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.000+0000 7fa326ffd700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa31003d5c0 0x7fa31003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:15.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.000+0000 7fa326ffd700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa31003d5c0 0x7fa31003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006fd0 tx=0x7fa31c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:15.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.005+0000 7fa324ff9700 1 -- 172.21.15.96:0/3325009697 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa318013070 con 0x7fa3281029a0 2024-04-25T17:01:15.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.207+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa32802e7f0 con 0x7fa3281029a0 2024-04-25T17:01:15.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.210+0000 7fa324ff9700 1 -- 172.21.15.96:0/3325009697 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa3180292c0 con 0x7fa3281029a0 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa31003d5c0 msgr2=0x7fa31003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa31003d5c0 0x7fa31003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006fd0 tx=0x7fa31c006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 msgr2=0x7fa32806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa32806c090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3180057d0 tx=0x7fa318005800 comp rx=0 tx=0).stop 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 shutdown_connections 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa31003d5c0 0x7fa31003fad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 --2- 172.21.15.96:0/3325009697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3281029a0 0x7fa32806c090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 >> 172.21.15.96:0/3325009697 conn(0x7fa3280fdfe0 msgr2=0x7fa3280fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 shutdown_connections 2024-04-25T17:01:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:15.212+0000 7fa32dccf700 1 -- 172.21.15.96:0/3325009697 wait complete. 2024-04-25T17:01:16.072 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T17:01:16.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T17:01:16.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:16 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3325009697' entity='client.admin' 2024-04-25T17:01:16.924 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:01:18.104 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:18.104 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:18.104 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:01:18.105 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:18.105 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:18.105 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:18.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.358+0000 7f247d7f1700 1 -- 172.21.15.96:0/1988156334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 msgr2=0x7f2478106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:18.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.358+0000 7f247d7f1700 1 --2- 172.21.15.96:0/1988156334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478106990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2468009910 tx=0x7f2468009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:18.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.358+0000 7f247d7f1700 1 -- 172.21.15.96:0/1988156334 shutdown_connections 2024-04-25T17:01:18.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.358+0000 7f247d7f1700 1 --2- 172.21.15.96:0/1988156334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478106990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:18.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.358+0000 7f247d7f1700 1 -- 172.21.15.96:0/1988156334 >> 172.21.15.96:0/1988156334 conn(0x7f24780fdfb0 msgr2=0x7f2478100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:18.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.358+0000 7f247d7f1700 1 -- 172.21.15.96:0/1988156334 shutdown_connections 2024-04-25T17:01:18.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.358+0000 7f247d7f1700 1 -- 172.21.15.96:0/1988156334 wait complete. 2024-04-25T17:01:18.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.359+0000 7f247d7f1700 1 Processor -- start 2024-04-25T17:01:18.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.359+0000 7f247d7f1700 1 -- start start 2024-04-25T17:01:18.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.359+0000 7f247d7f1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:18.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.359+0000 7f247d7f1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2478109a30 con 0x7f2478104570 2024-04-25T17:01:18.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.359+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.359+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48530/0 (socket says 172.21.15.96:48530) 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.359+0000 7f2476ffd700 1 -- 172.21.15.96:0/127453442 learned_addr learned my addr 172.21.15.96:0/127453442 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.360+0000 7f2476ffd700 1 -- 172.21.15.96:0/127453442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2478199ac0 con 0x7f2478104570 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.360+0000 7f2476ffd700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478199580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24680050d0 tx=0x7f2468009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.360+0000 7f2474ff9700 1 -- 172.21.15.96:0/127453442 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2468027070 con 0x7f2478104570 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.360+0000 7f2474ff9700 1 -- 172.21.15.96:0/127453442 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f246800be10 con 0x7f2478104570 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.360+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2478199d50 con 0x7f2478104570 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.360+0000 7f2474ff9700 1 -- 172.21.15.96:0/127453442 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2468022d80 con 0x7f2478104570 2024-04-25T17:01:18.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.360+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247819c960 con 0x7f2478104570 2024-04-25T17:01:18.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.361+0000 7f2474ff9700 1 -- 172.21.15.96:0/127453442 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f246802a070 con 0x7f2478104570 2024-04-25T17:01:18.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.361+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247810a1b0 con 0x7f2478104570 2024-04-25T17:01:18.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.361+0000 7f2474ff9700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f246403d670 0x7f246403fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:18.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.361+0000 7f2474ff9700 1 -- 172.21.15.96:0/127453442 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f246801e030 con 0x7f2478104570 2024-04-25T17:01:18.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.361+0000 7f246e7fc700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f246403d670 0x7f246403fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:18.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.362+0000 7f246e7fc700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f246403d670 0x7f246403fb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2460006fd0 tx=0x7f2460006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.367+0000 7f2474ff9700 1 -- 172.21.15.96:0/127453442 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2468013070 con 0x7f2478104570 2024-04-25T17:01:18.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.568+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f247802e620 con 0x7f246403d670 2024-04-25T17:01:18.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.572+0000 7f2474ff9700 1 -- 172.21.15.96:0/127453442 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f247802e620 con 0x7f246403d670 2024-04-25T17:01:18.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f246403d670 msgr2=0x7f246403fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f246403d670 0x7f246403fb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2460006fd0 tx=0x7f2460006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 msgr2=0x7f2478199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478199580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24680050d0 tx=0x7f2468009f80 comp rx=0 tx=0).stop 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 shutdown_connections 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f246403d670 0x7f246403fb80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 --2- 172.21.15.96:0/127453442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2478104570 0x7f2478199580 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 >> 172.21.15.96:0/127453442 conn(0x7f24780fdfb0 msgr2=0x7f24780fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.574+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 shutdown_connections 2024-04-25T17:01:18.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:18.575+0000 7f247d7f1700 1 -- 172.21.15.96:0/127453442 wait complete. 2024-04-25T17:01:19.338 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2024-04-25T17:01:19.339 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T17:01:19.339 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2024-04-25T17:01:19.357 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T17:01:19.357 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:19.414 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2024-04-25T17:01:19.414 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch host add smithi097 2024-04-25T17:01:19.647 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:01:19.891 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:19.891 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:19.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:20.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:20 smithi096 ceph-mon[29131]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:01:21.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.187+0000 7f58843e4700 1 -- 172.21.15.96:0/1431637270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 msgr2=0x7f587c104f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:21.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.187+0000 7f58843e4700 1 --2- 172.21.15.96:0/1431637270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c104f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5878009910 tx=0x7f5878009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:21.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.187+0000 7f58843e4700 1 -- 172.21.15.96:0/1431637270 shutdown_connections 2024-04-25T17:01:21.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.187+0000 7f58843e4700 1 --2- 172.21.15.96:0/1431637270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c104f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:21.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.187+0000 7f58843e4700 1 -- 172.21.15.96:0/1431637270 >> 172.21.15.96:0/1431637270 conn(0x7f587c0fe460 msgr2=0x7f587c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:21.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.187+0000 7f58843e4700 1 -- 172.21.15.96:0/1431637270 shutdown_connections 2024-04-25T17:01:21.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.187+0000 7f58843e4700 1 -- 172.21.15.96:0/1431637270 wait complete. 2024-04-25T17:01:21.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f58843e4700 1 Processor -- start 2024-04-25T17:01:21.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f58843e4700 1 -- start start 2024-04-25T17:01:21.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f58843e4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:21.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f58843e4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10dfd0 con 0x7f587c104b70 2024-04-25T17:01:21.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f5882180700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f5882180700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c1a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48562/0 (socket says 172.21.15.96:48562) 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f5882180700 1 -- 172.21.15.96:0/2953929313 learned_addr learned my addr 172.21.15.96:0/2953929313 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.188+0000 7f5882180700 1 -- 172.21.15.96:0/2953929313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c1a1b60 con 0x7f587c104b70 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.189+0000 7f5882180700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c1a1620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58780056d0 tx=0x7f58780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.189+0000 7f58737fe700 1 -- 172.21.15.96:0/2953929313 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5878026070 con 0x7f587c104b70 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.189+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c1a1df0 con 0x7f587c104b70 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.189+0000 7f58737fe700 1 -- 172.21.15.96:0/2953929313 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878004370 con 0x7f587c104b70 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.189+0000 7f58737fe700 1 -- 172.21.15.96:0/2953929313 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58780039b0 con 0x7f587c104b70 2024-04-25T17:01:21.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.189+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c19b860 con 0x7f587c104b70 2024-04-25T17:01:21.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.190+0000 7f58737fe700 1 -- 172.21.15.96:0/2953929313 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f5878029070 con 0x7f587c104b70 2024-04-25T17:01:21.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.190+0000 7f58737fe700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f586803d610 0x7f586803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:21.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.190+0000 7f58737fe700 1 -- 172.21.15.96:0/2953929313 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f587801e070 con 0x7f587c104b70 2024-04-25T17:01:21.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.190+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c06c290 con 0x7f587c104b70 2024-04-25T17:01:21.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.190+0000 7f588197f700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f586803d610 0x7f586803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:21.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.190+0000 7f588197f700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f586803d610 0x7f586803fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f586c006fd0 tx=0x7f586c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:21.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.195+0000 7f58737fe700 1 -- 172.21.15.96:0/2953929313 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5878030080 con 0x7f587c104b70 2024-04-25T17:01:21.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:21.396+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) -- 0x7f587c19cd70 con 0x7f586803d610 2024-04-25T17:01:22.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:22 smithi096 ceph-mon[29131]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:22.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:22 smithi096 ceph-mon[29131]: Deploying cephadm binary to smithi097 2024-04-25T17:01:23.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:23 smithi096 ceph-mon[29131]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:25.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.466+0000 7f58737fe700 1 -- 172.21.15.96:0/2953929313 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f587c19cd70 con 0x7f586803d610 2024-04-25T17:01:25.465 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi097' with addr '172.21.15.97' 2024-04-25T17:01:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.468+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f586803d610 msgr2=0x7f586803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.468+0000 7f58843e4700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f586803d610 0x7f586803fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f586c006fd0 tx=0x7f586c006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.468+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 msgr2=0x7f587c1a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.468+0000 7f58843e4700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c1a1620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58780056d0 tx=0x7f58780057b0 comp rx=0 tx=0).stop 2024-04-25T17:01:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.469+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 shutdown_connections 2024-04-25T17:01:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.469+0000 7f58843e4700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f586803d610 0x7f586803fb20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:25.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.469+0000 7f58843e4700 1 --2- 172.21.15.96:0/2953929313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f587c104b70 0x7f587c1a1620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.469+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 >> 172.21.15.96:0/2953929313 conn(0x7f587c0fe460 msgr2=0x7f587c109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.469+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 shutdown_connections 2024-04-25T17:01:25.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:25.469+0000 7f58843e4700 1 -- 172.21.15.96:0/2953929313 wait complete. 2024-04-25T17:01:26.029 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:26 smithi096 ceph-mon[29131]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:26.029 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:26.029 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:26.029 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:26.030 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T17:01:26.262 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:01:27.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:27 smithi096 ceph-mon[29131]: Added host smithi097 2024-04-25T17:01:27.256 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:27 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:27.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 -- 172.21.15.96:0/2093224036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 msgr2=0x7fd030107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 --2- 172.21.15.96:0/2093224036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd030107d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009910 tx=0x7fd02c009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 -- 172.21.15.96:0/2093224036 shutdown_connections 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 --2- 172.21.15.96:0/2093224036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd030107d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 -- 172.21.15.96:0/2093224036 >> 172.21.15.96:0/2093224036 conn(0x7fd0300fdd00 msgr2=0x7fd030100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 -- 172.21.15.96:0/2093224036 shutdown_connections 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 -- 172.21.15.96:0/2093224036 wait complete. 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 Processor -- start 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.717+0000 7fd03786c700 1 -- start start 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd03786c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd03019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd03786c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd03010ced0 con 0x7fd030107940 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd035608700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd03019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd035608700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd03019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51050/0 (socket says 172.21.15.96:51050) 2024-04-25T17:01:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd035608700 1 -- 172.21.15.96:0/1844539367 learned_addr learned my addr 172.21.15.96:0/1844539367 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd035608700 1 -- 172.21.15.96:0/1844539367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03019fa30 con 0x7fd030107940 2024-04-25T17:01:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd035608700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd03019f4f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00b5c0 tx=0x7fd02c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd0267fc700 1 -- 172.21.15.96:0/1844539367 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd02c026070 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.718+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03019fcc0 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.719+0000 7fd0267fc700 1 -- 172.21.15.96:0/1844539367 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c00b750 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.719+0000 7fd0267fc700 1 -- 172.21.15.96:0/1844539367 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd02c0037d0 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.719+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd030199680 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.719+0000 7fd0267fc700 1 -- 172.21.15.96:0/1844539367 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fd02c029070 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.720+0000 7fd0267fc700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd01c03d610 0x7fd01c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.720+0000 7fd0267fc700 1 -- 172.21.15.96:0/1844539367 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd02c01e070 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.720+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd030104050 con 0x7fd030107940 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.720+0000 7fd034e07700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd01c03d610 0x7fd01c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:27.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.720+0000 7fd034e07700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd01c03d610 0x7fd01c03fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd020006fd0 tx=0x7fd020006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:27.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.725+0000 7fd0267fc700 1 -- 172.21.15.96:0/1844539367 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd02c030020 con 0x7fd030107940 2024-04-25T17:01:27.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.936+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd03019ab90 con 0x7fd01c03d610 2024-04-25T17:01:27.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.937+0000 7fd0267fc700 1 -- 172.21.15.96:0/1844539367 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fd03019ab90 con 0x7fd01c03d610 2024-04-25T17:01:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:27.936 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2024-04-25T17:01:27.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd01c03d610 msgr2=0x7fd01c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:27.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd01c03d610 0x7fd01c03fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd020006fd0 tx=0x7fd020006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:27.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 msgr2=0x7fd03019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:27.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd03019f4f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00b5c0 tx=0x7fd02c005780 comp rx=0 tx=0).stop 2024-04-25T17:01:27.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 shutdown_connections 2024-04-25T17:01:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd01c03d610 0x7fd01c03fb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 --2- 172.21.15.96:0/1844539367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd030107940 0x7fd03019f4f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.939+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 >> 172.21.15.96:0/1844539367 conn(0x7fd0300fdd00 msgr2=0x7fd030100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.940+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 shutdown_connections 2024-04-25T17:01:27.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:27.940+0000 7fd03786c700 1 -- 172.21.15.96:0/1844539367 wait complete. 2024-04-25T17:01:28.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:28 smithi096 ceph-mon[29131]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:28.507 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-04-25T17:01:28.507 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T17:01:28.507 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-04-25T17:01:28.526 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T17:01:28.526 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:28.584 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-04-25T17:01:28.584 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch host add smithi171 2024-04-25T17:01:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:01:29.240 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:29 smithi096 ceph-mon[29131]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:01:29.240 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:30.147 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:30 smithi096 ceph-mon[29131]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:30.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.260+0000 7f2b0afcc700 1 -- 172.21.15.96:0/3387261278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 msgr2=0x7f2b04109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.260+0000 7f2b0afcc700 1 --2- 172.21.15.96:0/3387261278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04109e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009910 tx=0x7f2af4009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.260+0000 7f2b0afcc700 1 -- 172.21.15.96:0/3387261278 shutdown_connections 2024-04-25T17:01:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.260+0000 7f2b0afcc700 1 --2- 172.21.15.96:0/3387261278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04109e50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.260+0000 7f2b0afcc700 1 -- 172.21.15.96:0/3387261278 >> 172.21.15.96:0/3387261278 conn(0x7f2b040ff530 msgr2=0x7f2b04101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.260+0000 7f2b0afcc700 1 -- 172.21.15.96:0/3387261278 shutdown_connections 2024-04-25T17:01:30.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.260+0000 7f2b0afcc700 1 -- 172.21.15.96:0/3387261278 wait complete. 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b0afcc700 1 Processor -- start 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b0afcc700 1 -- start start 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b0afcc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b0afcc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0410df90 con 0x7f2b04109a40 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b08d68700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b08d68700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51066/0 (socket says 172.21.15.96:51066) 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b08d68700 1 -- 172.21.15.96:0/22100774 learned_addr learned my addr 172.21.15.96:0/22100774 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.261+0000 7f2b08d68700 1 -- 172.21.15.96:0/22100774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b040777e0 con 0x7f2b04109a40 2024-04-25T17:01:30.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.262+0000 7f2b08d68700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04075990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2af40056d0 tx=0x7f2af40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.262+0000 7f2b01ffb700 1 -- 172.21.15.96:0/22100774 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af4027070 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.262+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b04075ed0 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.262+0000 7f2b01ffb700 1 -- 172.21.15.96:0/22100774 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af400be10 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.262+0000 7f2b01ffb700 1 -- 172.21.15.96:0/22100774 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af4022d20 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.262+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b04076390 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.263+0000 7f2b01ffb700 1 -- 172.21.15.96:0/22100774 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f2af402a070 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.263+0000 7f2b01ffb700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2aec03d610 0x7f2aec03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.263+0000 7f2b01ffb700 1 -- 172.21.15.96:0/22100774 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2af401e030 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.263+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b04105150 con 0x7f2b04109a40 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.263+0000 7f2b03fff700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2aec03d610 0x7f2aec03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:30.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.264+0000 7f2b03fff700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2aec03d610 0x7f2aec03fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006fd0 tx=0x7f2af8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:30.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.268+0000 7f2b01ffb700 1 -- 172.21.15.96:0/22100774 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2af4031020 con 0x7f2b04109a40 2024-04-25T17:01:30.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:30.469+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) -- 0x7f2b04076610 con 0x7f2aec03d610 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: Deploying cephadm binary to smithi171 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:01:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:01:33.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:33 smithi096 ceph-mon[29131]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:01:33.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:33 smithi096 ceph-mon[29131]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:01:34.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:34 smithi096 ceph-mon[29131]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:34.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.571+0000 7f2b01ffb700 1 -- 172.21.15.96:0/22100774 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2b04076610 con 0x7f2aec03d610 2024-04-25T17:01:34.570 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2aec03d610 msgr2=0x7f2aec03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2aec03d610 0x7f2aec03fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2af8006fd0 tx=0x7f2af8006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 msgr2=0x7f2b04075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04075990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2af40056d0 tx=0x7f2af40057b0 comp rx=0 tx=0).stop 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 shutdown_connections 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2aec03d610 0x7f2aec03fb20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 --2- 172.21.15.96:0/22100774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b04109a40 0x7f2b04075990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 >> 172.21.15.96:0/22100774 conn(0x7f2b040ff530 msgr2=0x7f2b04100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 shutdown_connections 2024-04-25T17:01:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:34.573+0000 7f2b0afcc700 1 -- 172.21.15.96:0/22100774 wait complete. 2024-04-25T17:01:35.132 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:35 smithi096 ceph-mon[29131]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:35.132 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:35 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:35.132 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:35 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:35.132 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:35 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:35.132 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:35 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:35.133 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:35 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:35.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T17:01:35.405 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:01:36.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:36 smithi096 ceph-mon[29131]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:01:36.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:36 smithi096 ceph-mon[29131]: Added host smithi171 2024-04-25T17:01:36.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:36 smithi096 ceph-mon[29131]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:36.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:36.330 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:36.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.805+0000 7fe93480a700 1 -- 172.21.15.96:0/2568879868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 msgr2=0x7fe930108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:36.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.805+0000 7fe93480a700 1 --2- 172.21.15.96:0/2568879868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe930108d20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe920009910 tx=0x7fe920009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:36.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.805+0000 7fe93480a700 1 -- 172.21.15.96:0/2568879868 shutdown_connections 2024-04-25T17:01:36.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.805+0000 7fe93480a700 1 --2- 172.21.15.96:0/2568879868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe930108d20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:36.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.805+0000 7fe93480a700 1 -- 172.21.15.96:0/2568879868 >> 172.21.15.96:0/2568879868 conn(0x7fe9300fe420 msgr2=0x7fe930100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:36.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.805+0000 7fe93480a700 1 -- 172.21.15.96:0/2568879868 shutdown_connections 2024-04-25T17:01:36.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.805+0000 7fe93480a700 1 -- 172.21.15.96:0/2568879868 wait complete. 2024-04-25T17:01:36.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.806+0000 7fe93480a700 1 Processor -- start 2024-04-25T17:01:36.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.806+0000 7fe93480a700 1 -- start start 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.806+0000 7fe93480a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe9301986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.806+0000 7fe93480a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe93010ce60 con 0x7fe930108910 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.806+0000 7fe92e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe9301986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.806+0000 7fe92e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe9301986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51078/0 (socket says 172.21.15.96:51078) 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.806+0000 7fe92e59c700 1 -- 172.21.15.96:0/3928314074 learned_addr learned my addr 172.21.15.96:0/3928314074 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.807+0000 7fe92e59c700 1 -- 172.21.15.96:0/3928314074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe930198c30 con 0x7fe930108910 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.807+0000 7fe92e59c700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe9301986f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe9200056d0 tx=0x7fe9200057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.807+0000 7fe91f7fe700 1 -- 172.21.15.96:0/3928314074 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe920027070 con 0x7fe930108910 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.807+0000 7fe91f7fe700 1 -- 172.21.15.96:0/3928314074 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92000be10 con 0x7fe930108910 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.807+0000 7fe91f7fe700 1 -- 172.21.15.96:0/3928314074 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe920022d20 con 0x7fe930108910 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.807+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe930198f20 con 0x7fe930108910 2024-04-25T17:01:36.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.807+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93019cf20 con 0x7fe930108910 2024-04-25T17:01:36.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.808+0000 7fe91f7fe700 1 -- 172.21.15.96:0/3928314074 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fe92002a070 con 0x7fe930108910 2024-04-25T17:01:36.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.808+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe930104020 con 0x7fe930108910 2024-04-25T17:01:36.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.808+0000 7fe91f7fe700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe91803d670 0x7fe91803fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:36.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.808+0000 7fe91f7fe700 1 -- 172.21.15.96:0/3928314074 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe92001e030 con 0x7fe930108910 2024-04-25T17:01:36.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.808+0000 7fe92dd9b700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe91803d670 0x7fe91803fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:36.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.809+0000 7fe92dd9b700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe91803d670 0x7fe91803fb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe924006fd0 tx=0x7fe924006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:36.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:36.813+0000 7fe91f7fe700 1 -- 172.21.15.96:0/3928314074 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe920013070 con 0x7fe930108910 2024-04-25T17:01:37.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.041+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe930199a50 con 0x7fe91803d670 2024-04-25T17:01:37.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.042+0000 7fe91f7fe700 1 -- 172.21.15.96:0/3928314074 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fe930199a50 con 0x7fe91803d670 2024-04-25T17:01:37.040 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:01:37.040 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe91803d670 msgr2=0x7fe91803fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe91803d670 0x7fe91803fb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe924006fd0 tx=0x7fe924006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 msgr2=0x7fe9301986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe9301986f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe9200056d0 tx=0x7fe9200057b0 comp rx=0 tx=0).stop 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 shutdown_connections 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe91803d670 0x7fe91803fb80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 --2- 172.21.15.96:0/3928314074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe930108910 0x7fe9301986f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 >> 172.21.15.96:0/3928314074 conn(0x7fe9300fe420 msgr2=0x7fe9300ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 shutdown_connections 2024-04-25T17:01:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:37.044+0000 7fe93480a700 1 -- 172.21.15.96:0/3928314074 wait complete. 2024-04-25T17:01:37.646 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T17:01:37.646 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T17:01:37.877 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:01:38.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:38 smithi096 ceph-mon[29131]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:38.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:38 smithi096 ceph-mon[29131]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:01:38.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.441+0000 7f985cb28700 1 -- 172.21.15.96:0/1755824178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 msgr2=0x7f9858107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.441+0000 7f985cb28700 1 --2- 172.21.15.96:0/1755824178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f9858107d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f984c009910 tx=0x7f984c009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.441+0000 7f985cb28700 1 -- 172.21.15.96:0/1755824178 shutdown_connections 2024-04-25T17:01:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.441+0000 7f985cb28700 1 --2- 172.21.15.96:0/1755824178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f9858107d70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.441+0000 7f985cb28700 1 -- 172.21.15.96:0/1755824178 >> 172.21.15.96:0/1755824178 conn(0x7f98580fdd00 msgr2=0x7f9858100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.441+0000 7f985cb28700 1 -- 172.21.15.96:0/1755824178 shutdown_connections 2024-04-25T17:01:39.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.441+0000 7f985cb28700 1 -- 172.21.15.96:0/1755824178 wait complete. 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.442+0000 7f985cb28700 1 Processor -- start 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.442+0000 7f985cb28700 1 -- start start 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.442+0000 7f985cb28700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f98581997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.442+0000 7f985cb28700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f985810cef0 con 0x7f9858107960 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f985659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f98581997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f985659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f98581997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45956/0 (socket says 172.21.15.96:45956) 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f985659c700 1 -- 172.21.15.96:0/2008050706 learned_addr learned my addr 172.21.15.96:0/2008050706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f985659c700 1 -- 172.21.15.96:0/2008050706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9858199d10 con 0x7f9858107960 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f985659c700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f98581997d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b5c0 tx=0x7f984c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f98477fe700 1 -- 172.21.15.96:0/2008050706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f984c026070 con 0x7f9858107960 2024-04-25T17:01:39.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9858199fa0 con 0x7f9858107960 2024-04-25T17:01:39.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f98477fe700 1 -- 172.21.15.96:0/2008050706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f984c00b750 con 0x7f9858107960 2024-04-25T17:01:39.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f98477fe700 1 -- 172.21.15.96:0/2008050706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f984c0037d0 con 0x7f9858107960 2024-04-25T17:01:39.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.443+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985819e000 con 0x7f9858107960 2024-04-25T17:01:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.444+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9858104240 con 0x7f9858107960 2024-04-25T17:01:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.444+0000 7f98477fe700 1 -- 172.21.15.96:0/2008050706 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f984c029070 con 0x7f9858107960 2024-04-25T17:01:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.444+0000 7f98477fe700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f984003d610 0x7f984003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.445+0000 7f98477fe700 1 -- 172.21.15.96:0/2008050706 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f984c01e070 con 0x7f9858107960 2024-04-25T17:01:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.445+0000 7f9855d9b700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f984003d610 0x7f984003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:39.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.445+0000 7f9855d9b700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f984003d610 0x7f984003fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9848006fd0 tx=0x7f9848006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:39.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.450+0000 7f98477fe700 1 -- 172.21.15.96:0/2008050706 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f984c013070 con 0x7f9858107960 2024-04-25T17:01:39.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:39.654+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f985819ab30 con 0x7f9858107960 2024-04-25T17:01:40.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.067+0000 7f98477fe700 1 -- 172.21.15.96:0/2008050706 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f984c00e3e0 con 0x7f9858107960 2024-04-25T17:01:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f984003d610 msgr2=0x7f984003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f984003d610 0x7f984003fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9848006fd0 tx=0x7f9848006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 msgr2=0x7f98581997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f98581997d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b5c0 tx=0x7f984c005780 comp rx=0 tx=0).stop 2024-04-25T17:01:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 shutdown_connections 2024-04-25T17:01:40.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f984003d610 0x7f984003fb20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 --2- 172.21.15.96:0/2008050706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9858107960 0x7f98581997d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 >> 172.21.15.96:0/2008050706 conn(0x7f98580fdd00 msgr2=0x7f9858101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 shutdown_connections 2024-04-25T17:01:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:01:40.070+0000 7f985cb28700 1 -- 172.21.15.96:0/2008050706 wait complete. 2024-04-25T17:01:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-04-25T17:01:40.347 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:40 smithi096 ceph-mon[29131]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:40.347 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:40 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2008050706' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T17:01:40.643 INFO:tasks.cephadm:Adding mon.a on smithi096 2024-04-25T17:01:40.643 INFO:tasks.cephadm:Adding mon.b on smithi097 2024-04-25T17:01:40.643 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-04-25T17:01:40.643 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi171:172.21.15.171=c' 2024-04-25T17:01:40.874 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T17:01:40.925 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T17:01:41.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2008050706' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:41.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:01:42.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:42 smithi096 ceph-mon[29131]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:01:42.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:42 smithi096 ceph-mon[29131]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:01:42.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:42 smithi096 ceph-mon[29131]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:42.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:42 smithi096 ceph-mon[29131]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:42.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:42 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:42.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:42 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:42.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:42 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:42.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.517+0000 7fd775f79700 1 -- 172.21.15.171:0/3663424231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 msgr2=0x7fd770108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:42.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.517+0000 7fd775f79700 1 --2- 172.21.15.171:0/3663424231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd770108d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd760009910 tx=0x7fd760009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:42.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.517+0000 7fd775f79700 1 -- 172.21.15.171:0/3663424231 shutdown_connections 2024-04-25T17:01:42.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.517+0000 7fd775f79700 1 --2- 172.21.15.171:0/3663424231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd770108d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:42.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.517+0000 7fd775f79700 1 -- 172.21.15.171:0/3663424231 >> 172.21.15.171:0/3663424231 conn(0x7fd7700fdd00 msgr2=0x7fd770100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:42.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.517+0000 7fd775f79700 1 -- 172.21.15.171:0/3663424231 shutdown_connections 2024-04-25T17:01:42.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.518+0000 7fd775f79700 1 -- 172.21.15.171:0/3663424231 wait complete. 2024-04-25T17:01:42.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.518+0000 7fd775f79700 1 Processor -- start 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.518+0000 7fd775f79700 1 -- start start 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.519+0000 7fd775f79700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd7701a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.519+0000 7fd775f79700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd77010ce80 con 0x7fd770108930 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.519+0000 7fd76f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd7701a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.519+0000 7fd76f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd7701a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.171:38014/0 (socket says 172.21.15.171:38014) 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.519+0000 7fd76f7fe700 1 -- 172.21.15.171:0/1768178599 learned_addr learned my addr 172.21.15.171:0/1768178599 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.519+0000 7fd76f7fe700 1 -- 172.21.15.171:0/1768178599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7701a1c00 con 0x7fd770108930 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.519+0000 7fd76f7fe700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd7701a16c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7600092e0 tx=0x7fd760005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.520+0000 7fd76cff9700 1 -- 172.21.15.171:0/1768178599 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd760026070 con 0x7fd770108930 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.520+0000 7fd76cff9700 1 -- 172.21.15.171:0/1768178599 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd76000b6f0 con 0x7fd770108930 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.520+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7701a1ef0 con 0x7fd770108930 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.520+0000 7fd76cff9700 1 -- 172.21.15.171:0/1768178599 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7600037f0 con 0x7fd770108930 2024-04-25T17:01:42.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.520+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77019b8e0 con 0x7fd770108930 2024-04-25T17:01:42.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.521+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd770104180 con 0x7fd770108930 2024-04-25T17:01:42.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.521+0000 7fd76cff9700 1 -- 172.21.15.171:0/1768178599 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fd760029070 con 0x7fd770108930 2024-04-25T17:01:42.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.521+0000 7fd76cff9700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd75803d610 0x7fd75803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:42.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.521+0000 7fd76cff9700 1 -- 172.21.15.171:0/1768178599 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd76001e070 con 0x7fd770108930 2024-04-25T17:01:42.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.521+0000 7fd76effd700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd75803d610 0x7fd75803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:42.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.522+0000 7fd76effd700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd75803d610 0x7fd75803fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd764006fd0 tx=0x7fd764006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:42.524 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.527+0000 7fd76cff9700 1 -- 172.21.15.171:0/1768178599 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd760013070 con 0x7fd770108930 2024-04-25T17:01:42.731 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.734+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}) -- 0x7fd77019ce10 con 0x7fd75803d610 2024-04-25T17:01:42.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.741+0000 7fd76cff9700 1 -- 172.21.15.171:0/1768178599 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd77019ce10 con 0x7fd75803d610 2024-04-25T17:01:42.739 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd75803d610 msgr2=0x7fd75803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd75803d610 0x7fd75803fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd764006fd0 tx=0x7fd764006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 msgr2=0x7fd7701a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd7701a16c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7600092e0 tx=0x7fd760005780 comp rx=0 tx=0).stop 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 shutdown_connections 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd75803d610 0x7fd75803fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 --2- 172.21.15.171:0/1768178599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd770108930 0x7fd7701a16c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.743+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 >> 172.21.15.171:0/1768178599 conn(0x7fd7700fdd00 msgr2=0x7fd770104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.744+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 shutdown_connections 2024-04-25T17:01:42.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:42.744+0000 7fd775f79700 1 -- 172.21.15.171:0/1768178599 wait complete. 2024-04-25T17:01:43.312 DEBUG:teuthology.orchestra.run.smithi097:mon.b> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T17:01:43.314 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T17:01:43.317 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T17:01:43.317 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T17:01:43.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:01:43.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:43.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:43.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:43.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:01:43.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:43.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:01:43.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:44 smithi096 ceph-mon[29131]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:44.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:44 smithi096 ceph-mon[29131]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi171:172.21.15.171=c;count:3 2024-04-25T17:01:44.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:44 smithi096 ceph-mon[29131]: Deploying daemon mon.c on smithi171 2024-04-25T17:01:44.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:44 smithi096 ceph-mon[29131]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:44.818 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:01:46.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:46 smithi096 ceph-mon[29131]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:46.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:46 smithi171 systemd[1]: Starting Ceph mon.c for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:01:47.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.041+0000 7f7c34438700 1 -- 172.21.15.171:0/68627070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 msgr2=0x7f7c2c104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:47.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.041+0000 7f7c34438700 1 --2- 172.21.15.171:0/68627070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c104ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009910 tx=0x7f7c1c009c20 comp rx=0 tx=0).stop 2024-04-25T17:01:47.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.041+0000 7f7c34438700 1 -- 172.21.15.171:0/68627070 shutdown_connections 2024-04-25T17:01:47.041 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.041+0000 7f7c34438700 1 --2- 172.21.15.171:0/68627070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c104ff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:47.041 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.042+0000 7f7c34438700 1 -- 172.21.15.171:0/68627070 >> 172.21.15.171:0/68627070 conn(0x7f7c2c078460 msgr2=0x7f7c2c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:47.041 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.042+0000 7f7c34438700 1 -- 172.21.15.171:0/68627070 shutdown_connections 2024-04-25T17:01:47.041 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.042+0000 7f7c34438700 1 -- 172.21.15.171:0/68627070 wait complete. 2024-04-25T17:01:47.041 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.042+0000 7f7c34438700 1 Processor -- start 2024-04-25T17:01:47.041 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.043+0000 7f7c34438700 1 -- start start 2024-04-25T17:01:47.042 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.043+0000 7f7c34438700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.043+0000 7f7c34438700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10dfe0 con 0x7f7c2c104be0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.043+0000 7f7c321d4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.043+0000 7f7c321d4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.171:38032/0 (socket says 172.21.15.171:38032) 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.043+0000 7f7c321d4700 1 -- 172.21.15.171:0/1252081277 learned_addr learned my addr 172.21.15.171:0/1252081277 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.044+0000 7f7c321d4700 1 -- 172.21.15.171:0/1252081277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c19fa30 con 0x7f7c2c104be0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.044+0000 7f7c321d4700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c19f4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0056d0 tx=0x7f7c1c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.044+0000 7f7c237fe700 1 -- 172.21.15.171:0/1252081277 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c1c026070 con 0x7f7c2c104be0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.044+0000 7f7c237fe700 1 -- 172.21.15.171:0/1252081277 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1c00bc00 con 0x7f7c2c104be0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.044+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c19fcc0 con 0x7f7c2c104be0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.044+0000 7f7c237fe700 1 -- 172.21.15.171:0/1252081277 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c1c003920 con 0x7f7c2c104be0 2024-04-25T17:01:47.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.044+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c1996d0 con 0x7f7c2c104be0 2024-04-25T17:01:47.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.045+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2c06b500 con 0x7f7c2c104be0 2024-04-25T17:01:47.044 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.045+0000 7f7c237fe700 1 -- 172.21.15.171:0/1252081277 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f7c1c029070 con 0x7f7c2c104be0 2024-04-25T17:01:47.045 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.046+0000 7f7c237fe700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7c1803d610 0x7f7c1803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:47.045 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.046+0000 7f7c237fe700 1 -- 172.21.15.171:0/1252081277 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7c1c01e070 con 0x7f7c2c104be0 2024-04-25T17:01:47.045 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.046+0000 7f7c319d3700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7c1803d610 0x7f7c1803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:47.045 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.046+0000 7f7c319d3700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7c1803d610 0x7f7c1803fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7c28006fd0 tx=0x7f7c28006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:47.049 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.051+0000 7f7c237fe700 1 -- 172.21.15.171:0/1252081277 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c1c013070 con 0x7f7c2c104be0 2024-04-25T17:01:47.295 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 podman[33146]: 2024-04-25 17:01:47.034284326 +0000 UTC m=+0.097114143 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:01:47.295 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 podman[33146]: 2024-04-25 17:01:47.171658883 +0000 UTC m=+0.234488700 container create 005b74a688e831b7a55d7592ad95d1cbb9665b34d9bc3985aa7423f4f9f9a483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T17:01:47.335 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.338+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7c2c19ab90 con 0x7f7c2c104be0 2024-04-25T17:01:47.336 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.339+0000 7f7c237fe700 1 -- 172.21.15.171:0/1252081277 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f7c1c030300 con 0x7f7c2c104be0 2024-04-25T17:01:47.337 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T17:01:47.337 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","modified":"2024-04-25T17:00:05.529202Z","created":"2024-04-25T17:00:05.529202Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7c1803d610 msgr2=0x7f7c1803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7c1803d610 0x7f7c1803fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7c28006fd0 tx=0x7f7c28006e40 comp rx=0 tx=0).stop 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 msgr2=0x7f7c2c19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c19f4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0056d0 tx=0x7f7c1c0057b0 comp rx=0 tx=0).stop 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 shutdown_connections 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7c1803d610 0x7f7c1803fb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 --2- 172.21.15.171:0/1252081277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c2c104be0 0x7f7c2c19f4f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 >> 172.21.15.171:0/1252081277 conn(0x7f7c2c078460 msgr2=0x7f7c2c109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 shutdown_connections 2024-04-25T17:01:47.339 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:47.341+0000 7f7c34438700 1 -- 172.21.15.171:0/1252081277 wait complete. 2024-04-25T17:01:47.340 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 podman[33146]: 2024-04-25 17:01:47.358037632 +0000 UTC m=+0.420867453 container init 005b74a688e831b7a55d7592ad95d1cbb9665b34d9bc3985aa7423f4f9f9a483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 podman[33146]: 2024-04-25 17:01:47.361082423 +0000 UTC m=+0.423912242 container start 005b74a688e831b7a55d7592ad95d1cbb9665b34d9bc3985aa7423f4f9f9a483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pidfile_write: ignore empty --pid-file 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: load: jerasure load: lrc 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: RocksDB version: 7.9.2 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Git sha 0 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: DB SUMMARY 2024-04-25T17:01:47.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: DB Session ID: HH0MV42ETONWVJ0NLSB3 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: CURRENT file: CURRENT 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.error_if_exists: 0 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.create_if_missing: 0 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.paranoid_checks: 1 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.env: 0x564628476500 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.info_log: 0x564629e8dfc0 2024-04-25T17:01:47.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.statistics: (nil) 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.use_fsync: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_log_file_size: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.allow_fallocate: 1 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.use_direct_reads: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T17:01:47.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.db_log_dir: 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.wal_dir: 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.write_buffer_manager: 0x564629e5e8c0 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T17:01:47.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.unordered_write: 0 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.row_cache: None 2024-04-25T17:01:47.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.wal_filter: None 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.two_write_queues: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.wal_compression: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.atomic_flush: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.log_readahead_size: 0 2024-04-25T17:01:47.785 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_background_jobs: 2 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_background_compactions: -1 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_subcompactions: 1 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T17:01:47.786 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_open_files: -1 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_background_flushes: -1 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Compression algorithms supported: 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kZSTD supported: 0 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kXpressCompression supported: 0 2024-04-25T17:01:47.787 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kZlibCompression supported: 1 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kSnappyCompression supported: 1 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kLZ4Compression supported: 1 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: kBZip2Compression supported: 0 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.merge_operator: 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_filter: None 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T17:01:47.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564629f941a0) 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_type: 0 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_index_type: 0 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_shortening: 1 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: checksum: 4 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: no_block_cache: 0 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache: 0x564629e911f0 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_options: 2024-04-25T17:01:47.789 INFO:journalctl@ceph.mon.c.smithi171.stdout: capacity : 536870912 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_shard_bits : 4 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: strict_capacity_limit : 0 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_compressed: (nil) 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: persistent_cache: (nil) 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size: 4096 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size_deviation: 10 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_restart_interval: 16 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_block_restart_interval: 1 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: metadata_block_size: 4096 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: partition_filters: 0 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: use_delta_encoding: 1 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: filter_policy: bloomfilter 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: whole_key_filtering: 1 2024-04-25T17:01:47.790 INFO:journalctl@ceph.mon.c.smithi171.stdout: verify_compression: 0 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: format_version: 5 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_index_compression: 1 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_align: 0 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: max_auto_readahead_size: 262144 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: prepopulate_block_cache: 0 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: initial_auto_readahead_size: 8192 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression: NoCompression 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T17:01:47.791 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.num_levels: 7 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T17:01:47.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T17:01:47.793 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T17:01:47.794 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.inplace_update_support: 0 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T17:01:47.795 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.bloom_locality: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.max_successive_merges: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.ttl: 2592000 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.enable_blob_files: false 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.min_blob_size: 0 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T17:01:47.796 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a5833013-60ce-4f1e-af76-ca510001bcc1 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714064507402990, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714064507403413, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714064507, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a5833013-60ce-4f1e-af76-ca510001bcc1", "db_session_id": "HH0MV42ETONWVJ0NLSB3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714064507403494, "job": 1, "event": "recovery_finished"} 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T17:01:47.797 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564629e75c00 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: DB pointer 0x56462ec68000 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** DB Stats ** 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-25T17:01:47.798 INFO:journalctl@ceph.mon.c.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T17:01:47.799 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache BinnedLRUCache@0x564629e911f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 7e-06 secs_since: 0 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: starting mon.c rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:01:47.800 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(???) e0 preinit fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 bash[33146]: 005b74a688e831b7a55d7592ad95d1cbb9665b34d9bc3985aa7423f4f9f9a483 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 systemd[1]: Started Ceph mon.c for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T17:01:47.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Deploying cephadm binary to smithi171 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:01:47.803 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Added host smithi171 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2008050706' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2008050706' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.804 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:01:47.805 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi171:172.21.15.171=c;count:3 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: Deploying daemon mon.c on smithi171 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:47.806 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:47 smithi171 ceph-mon[33189]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T17:01:49.110 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T17:01:49.110 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T17:01:49.356 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: mon.a calling monitor election 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: mon.c calling monitor election 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:52.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: monmap epoch 2 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: last_changed 2024-04-25T17:01:47.435620+0000 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: created 2024-04-25T17:00:05.529202+0000 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: min_mon_release 19 (squid) 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: election_strategy: 1 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: fsmap 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: mgrmap e13: x(active, since 49s) 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: overall HEALTH_OK 2024-04-25T17:01:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:52.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: mon.a calling monitor election 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: mon.c calling monitor election 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: monmap epoch 2 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: last_changed 2024-04-25T17:01:47.435620+0000 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: created 2024-04-25T17:00:05.529202+0000 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: min_mon_release 19 (squid) 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: election_strategy: 1 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: fsmap 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: mgrmap e13: x(active, since 49s) 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: overall HEALTH_OK 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:53.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:53 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:53.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:53 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:01:53.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:53 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:53.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:53 smithi171 ceph-mon[33189]: Deploying daemon mon.b on smithi097 2024-04-25T17:01:53.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:53 smithi171 ceph-mon[33189]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:53.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:53 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:53.891 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.892+0000 7f1c013ff700 1 -- 172.21.15.171:0/255573524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1be40035c0 msgr2=0x7f1be4005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:53.891 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.892+0000 7f1c013ff700 1 --2- 172.21.15.171:0/255573524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1be40035c0 0x7f1be4005aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc06b260 tx=0x7f1bec00feb0 comp rx=0 tx=0).stop 2024-04-25T17:01:53.891 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.892+0000 7f1c013ff700 1 -- 172.21.15.171:0/255573524 shutdown_connections 2024-04-25T17:01:53.892 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.892+0000 7f1c013ff700 1 --2- 172.21.15.171:0/255573524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1be40035c0 0x7f1be4005aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:53.892 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.892+0000 7f1c013ff700 1 -- 172.21.15.171:0/255573524 >> 172.21.15.171:0/255573524 conn(0x7f1bfc0fe4b0 msgr2=0x7f1bfc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:53.892 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.892+0000 7f1c013ff700 1 -- 172.21.15.171:0/255573524 shutdown_connections 2024-04-25T17:01:53.892 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.892+0000 7f1c013ff700 1 -- 172.21.15.171:0/255573524 wait complete. 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.893+0000 7f1c013ff700 1 Processor -- start 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.893+0000 7f1c013ff700 1 -- start start 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1c013ff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 0x7f1bfc19fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1c013ff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bfc1a0430 0x7f1bfc1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1c013ff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc1120b0 con 0x7f1bfc1a0430 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1c013ff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1bfc111f30 con 0x7f1be40035c0 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfaffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 0x7f1bfc19fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:53.895 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfaffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 0x7f1bfc19fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:41138/0 (socket says 172.21.15.171:41138) 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfaffd700 1 -- 172.21.15.171:0/934862798 learned_addr learned my addr 172.21.15.171:0/934862798 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfaffd700 1 -- 172.21.15.171:0/934862798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 msgr2=0x7f1bfc19fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfaffd700 1 -- 172.21.15.171:0/934862798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 msgr2=0x7f1bfc19fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfaffd700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 0x7f1bfc19fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfa7fc700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bfc1a0430 0x7f1bfc1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.894+0000 7f1bfaffd700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 0x7f1bfc19fef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1bfa7fc700 1 -- 172.21.15.171:0/934862798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 msgr2=0x7f1bfc19fef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1bfa7fc700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 0x7f1bfc19fef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1bfa7fc700 1 -- 172.21.15.171:0/934862798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bfc1a0a30 con 0x7f1bfc1a0430 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1bfa7fc700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bfc1a0430 0x7f1bfc1a4720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000b4d0 tx=0x7f1bf000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:53.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1be3fff700 1 -- 172.21.15.171:0/934862798 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1bf0011630 con 0x7f1bfc1a0430 2024-04-25T17:01:53.897 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1be3fff700 1 -- 172.21.15.171:0/934862798 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf0011c80 con 0x7f1bfc1a0430 2024-04-25T17:01:53.897 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1be3fff700 1 -- 172.21.15.171:0/934862798 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1bf0010da0 con 0x7f1bfc1a0430 2024-04-25T17:01:53.897 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bfc1a4c60 con 0x7f1bfc1a0430 2024-04-25T17:01:53.897 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.895+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bfc1a51f0 con 0x7f1bfc1a0430 2024-04-25T17:01:53.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.896+0000 7f1be3fff700 1 -- 172.21.15.171:0/934862798 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f1bf0018020 con 0x7f1bfc1a0430 2024-04-25T17:01:53.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.896+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bfc06ba60 con 0x7f1bfc1a0430 2024-04-25T17:01:53.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.897+0000 7f1be3fff700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1be803d400 0x7f1be803f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:01:53.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.897+0000 7f1be3fff700 1 -- 172.21.15.171:0/934862798 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1bf0050f60 con 0x7f1bfc1a0430 2024-04-25T17:01:53.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.897+0000 7f1bfaffd700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1be803d400 0x7f1be803f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:01:53.898 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.897+0000 7f1bfaffd700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1be803d400 0x7f1be803f910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00fa60 tx=0x7f1bec005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:01:53.901 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:53.903+0000 7f1be3fff700 1 -- 172.21.15.171:0/934862798 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bf001f020 con 0x7f1bfc1a0430 2024-04-25T17:01:53.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:53 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:01:53.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:53 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:01:53.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:53 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:01:53.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:53 smithi096 ceph-mon[29131]: Deploying daemon mon.b on smithi097 2024-04-25T17:01:53.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:53 smithi096 ceph-mon[29131]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:53.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:53 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:01:54.179 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.180+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1bfc1a12e0 con 0x7f1bfc1a0430 2024-04-25T17:01:54.180 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.181+0000 7f1be3fff700 1 -- 172.21.15.171:0/934862798 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1bfc1a12e0 con 0x7f1bfc1a0430 2024-04-25T17:01:54.180 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T17:01:54.180 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","modified":"2024-04-25T17:01:47.435620Z","created":"2024-04-25T17:00:05.529202Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T17:01:54.182 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1be803d400 msgr2=0x7f1be803f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:54.182 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1be803d400 0x7f1be803f910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00fa60 tx=0x7f1bec005960 comp rx=0 tx=0).stop 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bfc1a0430 msgr2=0x7f1bfc1a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bfc1a0430 0x7f1bfc1a4720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000b4d0 tx=0x7f1bf000b890 comp rx=0 tx=0).stop 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 shutdown_connections 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1be40035c0 0x7f1bfc19fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1be803d400 0x7f1be803f910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 --2- 172.21.15.171:0/934862798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1bfc1a0430 0x7f1bfc1a4720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 >> 172.21.15.171:0/934862798 conn(0x7f1bfc0fe4b0 msgr2=0x7f1bfc100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 shutdown_connections 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:01:54.183+0000 7f1c013ff700 1 -- 172.21.15.171:0/934862798 wait complete. 2024-04-25T17:01:54.183 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-04-25T17:01:54.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:54 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/934862798' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T17:01:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:54 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/934862798' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T17:01:55.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:01:55 smithi171 ceph-mon[33189]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:55.833 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T17:01:55.833 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T17:01:55.837 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:01:55 smithi096 ceph-mon[29131]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:01:56.077 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:02:01.662 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: mon.a calling monitor election 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: mon.c calling monitor election 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: mon.b calling monitor election 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.663 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: monmap epoch 3 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: last_changed 2024-04-25T17:01:56.300785+0000 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: created 2024-04-25T17:00:05.529202+0000 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: min_mon_release 19 (squid) 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: election_strategy: 1 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: fsmap 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T17:02:01.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: mgrmap e13: x(active, since 58s) 2024-04-25T17:02:01.665 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: overall HEALTH_OK 2024-04-25T17:02:01.665 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.665 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.665 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.665 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: mon.a calling monitor election 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: mon.c calling monitor election 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: mon.b calling monitor election 2024-04-25T17:02:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: monmap epoch 3 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: last_changed 2024-04-25T17:01:56.300785+0000 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: created 2024-04-25T17:00:05.529202+0000 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: min_mon_release 19 (squid) 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: election_strategy: 1 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-25T17:02:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: fsmap 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: mgrmap e13: x(active, since 58s) 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: overall HEALTH_OK 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:01.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.960+0000 7fba34843700 1 -- 172.21.15.171:0/3426133152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba180057d0 msgr2=0x7fba18007cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:01.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.960+0000 7fba34843700 1 --2- 172.21.15.171:0/3426133152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba180057d0 0x7fba18007cb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fba30102920 tx=0x7fba2400fe60 comp rx=0 tx=0).stop 2024-04-25T17:02:01.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.961+0000 7fba34843700 1 -- 172.21.15.171:0/3426133152 shutdown_connections 2024-04-25T17:02:01.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.961+0000 7fba34843700 1 --2- 172.21.15.171:0/3426133152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba180057d0 0x7fba18007cb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:01.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.961+0000 7fba34843700 1 -- 172.21.15.171:0/3426133152 >> 172.21.15.171:0/3426133152 conn(0x7fba300fe530 msgr2=0x7fba30100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:01.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.961+0000 7fba34843700 1 -- 172.21.15.171:0/3426133152 shutdown_connections 2024-04-25T17:02:01.960 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.961+0000 7fba34843700 1 -- 172.21.15.171:0/3426133152 wait complete. 2024-04-25T17:02:01.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.962+0000 7fba34843700 1 Processor -- start 2024-04-25T17:02:01.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.962+0000 7fba34843700 1 -- start start 2024-04-25T17:02:01.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba34843700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 0x7fba301a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:01.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba34843700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 0x7fba3019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:01.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba34843700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba3019c090 0x7fba3019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:01.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba34843700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba30109890 con 0x7fba3019c090 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba34843700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba3006cb20 con 0x7fba301a1f00 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba34843700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba30109a10 con 0x7fba180057d0 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2e59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 0x7fba301a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2e59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 0x7fba301a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:50806/0 (socket says 172.21.15.171:50806) 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2e59c700 1 -- 172.21.15.171:0/3578594284 learned_addr learned my addr 172.21.15.171:0/3578594284 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2dd9b700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 0x7fba3019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2e59c700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 msgr2=0x7fba301a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2e59c700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 msgr2=0x7fba301a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2e59c700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 0x7fba301a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2e59c700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 0x7fba301a19c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2dd9b700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 msgr2=0x7fba3019bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T17:02:01.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2dd9b700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 msgr2=0x7fba3019bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2dd9b700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 0x7fba3019bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2ed9d700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba3019c090 0x7fba3019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.963+0000 7fba2dd9b700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 0x7fba3019bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba2ed9d700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 msgr2=0x7fba301a19c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba2ed9d700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 0x7fba301a19c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba2ed9d700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 msgr2=0x7fba3019bb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba2ed9d700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 0x7fba3019bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba2ed9d700 1 -- 172.21.15.171:0/3578594284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba301a8770 con 0x7fba3019c090 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba2ed9d700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba3019c090 0x7fba3019c4a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fba2800bd50 tx=0x7fba2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:01.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba1f7fe700 1 -- 172.21.15.171:0/3578594284 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28019070 con 0x7fba3019c090 2024-04-25T17:02:01.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba1f7fe700 1 -- 172.21.15.171:0/3578594284 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba2800cd10 con 0x7fba3019c090 2024-04-25T17:02:01.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba301a8a60 con 0x7fba3019c090 2024-04-25T17:02:01.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba1f7fe700 1 -- 172.21.15.171:0/3578594284 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28012700 con 0x7fba3019c090 2024-04-25T17:02:01.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.964+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba301a8fc0 con 0x7fba3019c090 2024-04-25T17:02:01.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.965+0000 7fba1f7fe700 1 -- 172.21.15.171:0/3578594284 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fba2801c070 con 0x7fba3019c090 2024-04-25T17:02:01.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.966+0000 7fba1f7fe700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fba1003daa0 0x7fba1003ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:01.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.966+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba30104130 con 0x7fba3019c090 2024-04-25T17:02:01.967 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.966+0000 7fba1f7fe700 1 -- 172.21.15.171:0/3578594284 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fba28051e80 con 0x7fba3019c090 2024-04-25T17:02:01.967 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.966+0000 7fba2e59c700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fba1003daa0 0x7fba1003ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:01.967 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.966+0000 7fba2e59c700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fba1003daa0 0x7fba1003ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba2400f540 tx=0x7fba2400f890 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:01.970 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:01.971+0000 7fba1f7fe700 1 -- 172.21.15.171:0/3578594284 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba280112a0 con 0x7fba3019c090 2024-04-25T17:02:02.326 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.326+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fba3002e510 con 0x7fba3019c090 2024-04-25T17:02:02.326 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.327+0000 7fba1f7fe700 1 -- 172.21.15.171:0/3578594284 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fba28022c30 con 0x7fba3019c090 2024-04-25T17:02:02.327 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-25T17:02:02.327 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","modified":"2024-04-25T17:01:56.300785Z","created":"2024-04-25T17:00:05.529202Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fba1003daa0 msgr2=0x7fba1003ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fba1003daa0 0x7fba1003ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba2400f540 tx=0x7fba2400f890 comp rx=0 tx=0).stop 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba3019c090 msgr2=0x7fba3019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba3019c090 0x7fba3019c4a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fba2800bd50 tx=0x7fba2800be30 comp rx=0 tx=0).stop 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 shutdown_connections 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fba1003daa0 0x7fba1003ffb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba3019c090 0x7fba3019c4a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba180057d0 0x7fba301a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 --2- 172.21.15.171:0/3578594284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba301a1f00 0x7fba3019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 >> 172.21.15.171:0/3578594284 conn(0x7fba300fe530 msgr2=0x7fba30106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:02.329 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 shutdown_connections 2024-04-25T17:02:02.330 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:02.330+0000 7fba34843700 1 -- 172.21.15.171:0/3578594284 wait complete. 2024-04-25T17:02:02.330 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-04-25T17:02:02.640 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:02 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:02.640 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:02 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:02.640 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:02 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/3578594284' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T17:02:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:02 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:02 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:02 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/3578594284' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T17:02:02.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:02 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:02.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:02 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:02:02.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:02 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/3578594284' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T17:02:02.979 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T17:02:02.979 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T17:02:03.225 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:02:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.028 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.028 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.028 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:04.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:04.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T17:02:04.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:02:04.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:02:04.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:04.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T17:02:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:04 smithi096 ceph-mon[29131]: Reconfiguring daemon mon.a on smithi096 2024-04-25T17:02:05.028 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T17:02:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:04 smithi171 ceph-mon[33189]: Reconfiguring daemon mon.a on smithi096 2024-04-25T17:02:05.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T17:02:05.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:02:05.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:02:05.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:05.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:05.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:04 smithi097 ceph-mon[32602]: Reconfiguring daemon mon.a on smithi096 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.109+0000 7f636a986700 1 -- 172.21.15.96:0/2136969357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364104c50 msgr2=0x7f6364105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.109+0000 7f636a986700 1 --2- 172.21.15.96:0/2136969357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364104c50 0x7f6364105060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6354009910 tx=0x7f6354009c20 comp rx=0 tx=0).stop 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.109+0000 7f636a986700 1 -- 172.21.15.96:0/2136969357 shutdown_connections 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.109+0000 7f636a986700 1 --2- 172.21.15.96:0/2136969357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364104c50 0x7f6364105060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.109+0000 7f636a986700 1 -- 172.21.15.96:0/2136969357 >> 172.21.15.96:0/2136969357 conn(0x7f63640fe480 msgr2=0x7f63641008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.109+0000 7f636a986700 1 -- 172.21.15.96:0/2136969357 shutdown_connections 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.109+0000 7f636a986700 1 -- 172.21.15.96:0/2136969357 wait complete. 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 Processor -- start 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 -- start start 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6364104c50 0x7f636419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:05.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636419fcd0 0x7f63641999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364199ef0 0x7f636419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f636410e040 con 0x7f6364199ef0 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f636410dec0 con 0x7f636419fcd0 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f636a986700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f636410e1c0 con 0x7f6364104c50 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f6368f23700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364199ef0 0x7f636419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f6368f23700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364199ef0 0x7f636419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51204/0 (socket says 172.21.15.96:51204) 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.110+0000 7f6368f23700 1 -- 172.21.15.96:0/2588231995 learned_addr learned my addr 172.21.15.96:0/2588231995 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f63637fe700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636419fcd0 0x7f63641999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6363fff700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6364104c50 0x7f636419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6368f23700 1 -- 172.21.15.96:0/2588231995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6364104c50 msgr2=0x7f636419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6368f23700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6364104c50 0x7f636419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6368f23700 1 -- 172.21.15.96:0/2588231995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636419fcd0 msgr2=0x7f63641999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6368f23700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636419fcd0 0x7f63641999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:05.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6368f23700 1 -- 172.21.15.96:0/2588231995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63641a6620 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6363fff700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6364104c50 0x7f636419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f6368f23700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364199ef0 0x7f636419a300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f635c00ba50 tx=0x7f635c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f63617fa700 1 -- 172.21.15.96:0/2588231995 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c019070 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6364102810 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f63617fa700 1 -- 172.21.15.96:0/2588231995 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f635c014440 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f63617fa700 1 -- 172.21.15.96:0/2588231995 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c007590 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.111+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6364102d50 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.112+0000 7f63617fa700 1 -- 172.21.15.96:0/2588231995 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f635c01c070 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.112+0000 7f63617fa700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f634c03daa0 0x7f634c03ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.112+0000 7f63617fa700 1 -- 172.21.15.96:0/2588231995 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f635c051c00 con 0x7f6364199ef0 2024-04-25T17:02:05.521 INFO:teuthology.orchestra.run.smithi096.stdout:# minimal ceph.conf for 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stdout: mon_host = [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.113+0000 7f6363fff700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f634c03daa0 0x7f634c03ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.113+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6364064cf0 con 0x7f6364199ef0 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.113+0000 7f6363fff700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f634c03daa0 0x7f634c03ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6354006010 tx=0x7f635400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.118+0000 7f63617fa700 1 -- 172.21.15.96:0/2588231995 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f635c01bcd0 con 0x7f6364199ef0 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.316+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6364103070 con 0x7f6364199ef0 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.316+0000 7f63617fa700 1 -- 172.21.15.96:0/2588231995 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f635c014910 con 0x7f6364199ef0 2024-04-25T17:02:05.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f634c03daa0 msgr2=0x7f634c03ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f634c03daa0 0x7f634c03ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6354006010 tx=0x7f635400b540 comp rx=0 tx=0).stop 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364199ef0 msgr2=0x7f636419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364199ef0 0x7f636419a300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f635c00ba50 tx=0x7f635c00be10 comp rx=0 tx=0).stop 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 shutdown_connections 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f634c03daa0 0x7f634c03ffb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6364199ef0 0x7f636419a300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6364104c50 0x7f636419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 --2- 172.21.15.96:0/2588231995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f636419fcd0 0x7f63641999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 >> 172.21.15.96:0/2588231995 conn(0x7f63640fe480 msgr2=0x7f636410a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 shutdown_connections 2024-04-25T17:02:05.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:05.318+0000 7f636a986700 1 -- 172.21.15.96:0/2588231995 wait complete. 2024-04-25T17:02:06.055 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:05 smithi096 ceph-mon[29131]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:06.055 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:05 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2588231995' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:06.087 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T17:02:06.088 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:02:06.088 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T17:02:06.116 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:02:06.116 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:02:06.181 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T17:02:06.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T17:02:06.207 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:05 smithi097 ceph-mon[32602]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:06.208 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:05 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2588231995' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:06.213 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T17:02:06.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:02:06.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:05 smithi171 ceph-mon[33189]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:06.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:05 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2588231995' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:06.281 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T17:02:06.281 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T17:02:06.313 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T17:02:06.313 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:02:06.380 INFO:tasks.cephadm:Adding mgr.x on smithi096 2024-04-25T17:02:06.381 INFO:tasks.cephadm:Adding mgr.y on smithi097 2024-04-25T17:02:06.381 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-04-25T17:02:06.381 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi096=x;smithi097=y;smithi171=z' 2024-04-25T17:02:06.662 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:02:07.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:07.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:07.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: Reconfiguring mon.b (monmap changed)... 2024-04-25T17:02:07.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:07.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:07.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:07.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: Reconfiguring daemon mon.b on smithi097 2024-04-25T17:02:07.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:07 smithi171 ceph-mon[33189]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: Reconfiguring mon.b (monmap changed)... 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: Reconfiguring daemon mon.b on smithi097 2024-04-25T17:02:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:07 smithi096 ceph-mon[29131]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:07.979 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:07.979 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:07.980 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: Reconfiguring mon.b (monmap changed)... 2024-04-25T17:02:07.980 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:07.980 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:07.980 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:07.980 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: Reconfiguring daemon mon.b on smithi097 2024-04-25T17:02:07.980 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:07 smithi097 ceph-mon[32602]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:08.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.075+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/3179868946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08104730 msgr2=0x7fdf08106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:08.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.075+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/3179868946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08104730 0x7fdf08106b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009910 tx=0x7fdef0009c20 comp rx=0 tx=0).stop 2024-04-25T17:02:08.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.075+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/3179868946 shutdown_connections 2024-04-25T17:02:08.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.075+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/3179868946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08104730 0x7fdf08106b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:08.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.075+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/3179868946 >> 172.21.15.171:0/3179868946 conn(0x7fdf080fe0f0 msgr2=0x7fdf081005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:08.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.075+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/3179868946 shutdown_connections 2024-04-25T17:02:08.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.075+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/3179868946 wait complete. 2024-04-25T17:02:08.075 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 Processor -- start 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 -- start start 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf08104730 0x7fdf08103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf08102640 0x7fdf08102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08102f90 0x7fdf081a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08109b70 con 0x7fdf08102f90 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081099f0 con 0x7fdf08102640 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf0d9cb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08109cf0 con 0x7fdf08104730 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf06ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf08104730 0x7fdf08103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf06ffd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf08104730 0x7fdf08103f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:50814/0 (socket says 172.21.15.171:50814) 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf06ffd700 1 -- 172.21.15.171:0/410327142 learned_addr learned my addr 172.21.15.171:0/410327142 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf077fe700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08102f90 0x7fdf081a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:08.076 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.076+0000 7fdf067fc700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf08102640 0x7fdf08102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf077fe700 1 -- 172.21.15.171:0/410327142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf08104730 msgr2=0x7fdf08103f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf077fe700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf08104730 0x7fdf08103f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf077fe700 1 -- 172.21.15.171:0/410327142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf08102640 msgr2=0x7fdf08102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf077fe700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf08102640 0x7fdf08102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf077fe700 1 -- 172.21.15.171:0/410327142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf081a3e40 con 0x7fdf08102f90 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf077fe700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08102f90 0x7fdf081a3900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000bd50 tx=0x7fdf0000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf0c9c9700 1 -- 172.21.15.171:0/410327142 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00019070 con 0x7fdf08102f90 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081a40d0 con 0x7fdf08102f90 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf0c9c9700 1 -- 172.21.15.171:0/410327142 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0000cbb0 con 0x7fdf08102f90 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf0c9c9700 1 -- 172.21.15.171:0/410327142 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf000125a0 con 0x7fdf08102f90 2024-04-25T17:02:08.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.077+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081a4660 con 0x7fdf08102f90 2024-04-25T17:02:08.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.078+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf08064cf0 con 0x7fdf08102f90 2024-04-25T17:02:08.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.078+0000 7fdf0c9c9700 1 -- 172.21.15.171:0/410327142 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fdf0001c070 con 0x7fdf08102f90 2024-04-25T17:02:08.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.079+0000 7fdf0c9c9700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdef403daa0 0x7fdef403ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:08.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.079+0000 7fdf0c9c9700 1 -- 172.21.15.171:0/410327142 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdf00050d30 con 0x7fdf08102f90 2024-04-25T17:02:08.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.079+0000 7fdf06ffd700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdef403daa0 0x7fdef403ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:08.078 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.079+0000 7fdf06ffd700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdef403daa0 0x7fdef403ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdef0006010 tx=0x7fdef001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:08.083 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.086+0000 7fdf0c9c9700 1 -- 172.21.15.171:0/410327142 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf0001ce10 con 0x7fdf08102f90 2024-04-25T17:02:08.305 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.307+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi171=z", "target": ["mon-mgr", ""]}) -- 0x7fdf080fff30 con 0x7fdef403daa0 2024-04-25T17:02:08.312 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.314+0000 7fdf0c9c9700 1 -- 172.21.15.171:0/410327142 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdf080fff30 con 0x7fdef403daa0 2024-04-25T17:02:08.312 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.317+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdef403daa0 msgr2=0x7fdef403ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.317+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdef403daa0 0x7fdef403ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdef0006010 tx=0x7fdef001a040 comp rx=0 tx=0).stop 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.317+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08102f90 msgr2=0x7fdf081a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.317+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08102f90 0x7fdf081a3900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000bd50 tx=0x7fdf0000bd80 comp rx=0 tx=0).stop 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 shutdown_connections 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdef403daa0 0x7fdef403ffb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf08102f90 0x7fdf081a3900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf08104730 0x7fdf08103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 --2- 172.21.15.171:0/410327142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf08102640 0x7fdf08102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 >> 172.21.15.171:0/410327142 conn(0x7fdf080fe0f0 msgr2=0x7fdf080ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:08.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 shutdown_connections 2024-04-25T17:02:08.317 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:02:08.318+0000 7fdf0d9cb700 1 -- 172.21.15.171:0/410327142 wait complete. 2024-04-25T17:02:08.965 DEBUG:teuthology.orchestra.run.smithi097:mgr.y> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T17:02:08.967 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T17:02:08.969 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T17:02:08.970 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:02:08.970 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T17:02:08.989 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T17:02:08.989 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-04-25T17:02:09.047 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T17:02:09.047 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:09.047 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T17:02:09.048 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:09.048 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:09.048 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 17:01:16.020801197 +0000 2024-04-25T17:02:09.048 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 16:56:28.404584093 +0000 2024-04-25T17:02:09.048 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 16:56:28.404584093 +0000 2024-04-25T17:02:09.048 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 16:56:28.404584093 +0000 2024-04-25T17:02:09.048 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T17:02:09.114 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T17:02:09.114 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T17:02:09.114 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000228032 s, 2.2 MB/s 2024-04-25T17:02:09.115 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T17:02:09.173 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-04-25T17:02:09.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: Reconfiguring mon.c (monmap changed)... 2024-04-25T17:02:09.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:09.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:09.217 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:09.217 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: Reconfiguring daemon mon.c on smithi171 2024-04-25T17:02:09.217 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T17:02:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:09.230 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T17:02:09.230 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:09.230 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:09.230 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 17:01:16.046800628 +0000 2024-04-25T17:02:09.230 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 16:56:28.760575723 +0000 2024-04-25T17:02:09.230 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 16:56:28.760575723 +0000 2024-04-25T17:02:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 16:56:28.760575723 +0000 2024-04-25T17:02:09.230 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T17:02:09.292 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.292 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.292 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: Reconfiguring mon.c (monmap changed)... 2024-04-25T17:02:09.292 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:09.292 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:09.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:09.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: Reconfiguring daemon mon.c on smithi171 2024-04-25T17:02:09.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T17:02:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T17:02:09.295 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000235674 s, 2.2 MB/s 2024-04-25T17:02:09.296 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T17:02:09.354 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-04-25T17:02:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T17:02:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:09.410 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T17:02:09.410 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:09.410 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:09.410 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 17:01:16.073800037 +0000 2024-04-25T17:02:09.411 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 16:56:29.097567800 +0000 2024-04-25T17:02:09.411 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 16:56:29.097567800 +0000 2024-04-25T17:02:09.411 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 16:56:29.097567800 +0000 2024-04-25T17:02:09.411 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T17:02:09.474 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T17:02:09.474 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T17:02:09.474 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000248475 s, 2.1 MB/s 2024-04-25T17:02:09.475 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: Reconfiguring mon.c (monmap changed)... 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: Reconfiguring daemon mon.c on smithi171 2024-04-25T17:02:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:09.531 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-04-25T17:02:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T17:02:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:09.587 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T17:02:09.587 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:09.587 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:09.587 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 17:01:16.100799447 +0000 2024-04-25T17:02:09.587 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 16:56:29.431559948 +0000 2024-04-25T17:02:09.587 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 16:56:29.431559948 +0000 2024-04-25T17:02:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 16:56:29.431559948 +0000 2024-04-25T17:02:09.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T17:02:09.649 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T17:02:09.649 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T17:02:09.649 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000231643 s, 2.2 MB/s 2024-04-25T17:02:09.650 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T17:02:09.707 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T17:02:09.707 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T17:02:09.726 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T17:02:09.727 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-04-25T17:02:09.784 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 17:01:55.730189845 +0000 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 16:56:59.075168253 +0000 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 16:56:59.075168253 +0000 2024-04-25T17:02:09.785 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 16:56:59.075168253 +0000 2024-04-25T17:02:09.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T17:02:09.850 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T17:02:09.851 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T17:02:09.851 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000288457 s, 1.8 MB/s 2024-04-25T17:02:09.852 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T17:02:09.910 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 17:01:55.731189828 +0000 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 16:56:59.405162307 +0000 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 16:56:59.405162307 +0000 2024-04-25T17:02:09.967 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 16:56:59.405162307 +0000 2024-04-25T17:02:09.968 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T17:02:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T17:02:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T17:02:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000228873 s, 2.2 MB/s 2024-04-25T17:02:10.034 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T17:02:10.091 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 17:01:55.731189828 +0000 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 16:56:59.794155299 +0000 2024-04-25T17:02:10.149 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 16:56:59.794155299 +0000 2024-04-25T17:02:10.150 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 16:56:59.794155299 +0000 2024-04-25T17:02:10.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T17:02:10.212 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:10 smithi097 ceph-mon[32602]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:10.212 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:10 smithi097 ceph-mon[32602]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi171=z;count:3 2024-04-25T17:02:10.212 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:10 smithi097 ceph-mon[32602]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:10.214 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T17:02:10.214 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T17:02:10.214 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000221792 s, 2.3 MB/s 2024-04-25T17:02:10.215 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T17:02:10.273 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-04-25T17:02:10.330 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 17:01:55.732189812 +0000 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 16:57:00.139149083 +0000 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 16:57:00.139149083 +0000 2024-04-25T17:02:10.331 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 16:57:00.139149083 +0000 2024-04-25T17:02:10.331 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T17:02:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T17:02:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T17:02:10.396 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000252314 s, 2.0 MB/s 2024-04-25T17:02:10.397 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T17:02:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:10 smithi096 ceph-mon[29131]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:10.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:10 smithi096 ceph-mon[29131]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi171=z;count:3 2024-04-25T17:02:10.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:10 smithi096 ceph-mon[29131]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:10.455 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-25T17:02:10.455 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T17:02:10.471 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T17:02:10.471 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-04-25T17:02:10.526 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T17:02:10.526 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:10.526 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T17:02:10.526 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:10.527 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:10.527 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 17:01:46.266152633 +0000 2024-04-25T17:02:10.527 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 16:56:47.120682394 +0000 2024-04-25T17:02:10.527 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 16:56:47.120682394 +0000 2024-04-25T17:02:10.527 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 16:56:47.120682394 +0000 2024-04-25T17:02:10.527 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T17:02:10.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:10 smithi171 ceph-mon[33189]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:10.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:10 smithi171 ceph-mon[33189]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi171=z;count:3 2024-04-25T17:02:10.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:10 smithi171 ceph-mon[33189]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:10.551 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T17:02:10.551 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T17:02:10.551 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000274438 s, 1.9 MB/s 2024-04-25T17:02:10.552 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T17:02:10.608 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 17:01:46.266152633 +0000 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 16:56:47.460675114 +0000 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 16:56:47.460675114 +0000 2024-04-25T17:02:10.663 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 16:56:47.460675114 +0000 2024-04-25T17:02:10.664 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T17:02:10.725 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T17:02:10.725 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T17:02:10.725 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000202418 s, 2.5 MB/s 2024-04-25T17:02:10.726 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T17:02:10.783 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 17:01:46.267152613 +0000 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 16:56:47.803667769 +0000 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 16:56:47.803667769 +0000 2024-04-25T17:02:10.838 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 16:56:47.803667769 +0000 2024-04-25T17:02:10.839 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T17:02:10.903 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T17:02:10.903 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T17:02:10.903 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000168883 s, 3.0 MB/s 2024-04-25T17:02:10.903 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T17:02:10.958 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-04-25T17:02:11.013 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-25 17:01:46.267152613 +0000 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-25 16:56:48.140660553 +0000 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-25 16:56:48.140660553 +0000 2024-04-25T17:02:11.014 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-25 16:56:48.140660553 +0000 2024-04-25T17:02:11.014 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T17:02:11.077 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-25T17:02:11.077 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-25T17:02:11.077 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000320144 s, 1.6 MB/s 2024-04-25T17:02:11.078 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T17:02:11.134 INFO:tasks.cephadm:Deploying osd.0 on smithi096 with /dev/vg_nvme/lv_4... 2024-04-25T17:02:11.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T17:02:11.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:11.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:11.362 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T17:02:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:11.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:11.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:11.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:11.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:11.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T17:02:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:12.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:12 smithi096 ceph-mon[29131]: Deploying daemon mgr.z on smithi171 2024-04-25T17:02:12.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:12 smithi096 ceph-mon[29131]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:12.329 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:12 smithi171 ceph-mon[33189]: Deploying daemon mgr.z on smithi171 2024-04-25T17:02:12.329 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:12 smithi171 ceph-mon[33189]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:12.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:12 smithi097 ceph-mon[32602]: Deploying daemon mgr.z on smithi171 2024-04-25T17:02:12.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:12 smithi097 ceph-mon[32602]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:13.097 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T17:02:13.097 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T17:02:13.097 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T17:02:13.097 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T17:02:13.097 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114817 s, 913 MB/s 2024-04-25T17:02:13.097 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T17:02:13.278 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:13 smithi171 systemd[1]: Started Ceph mgr.z for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:02:13.563 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:13.281+0000 7fdc1a091200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:02:13.776 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:13.464+0000 7fdc1a091200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:02:13.776 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:13.564+0000 7fdc1a091200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:02:13.814 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-04-25T17:02:14.045 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:02:14.190 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:14.190 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.190 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.190 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.190 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:14.191 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T17:02:14.191 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:14.191 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:14.278 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:13.870+0000 7fdc1a091200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:02:14.278 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:13.954+0000 7fdc1a091200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:14.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:15.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:15 smithi096 ceph-mon[29131]: Deploying daemon mgr.y on smithi097 2024-04-25T17:02:15.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:15 smithi097 ceph-mon[32602]: Deploying daemon mgr.y on smithi097 2024-04-25T17:02:15.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 systemd[1]: Starting Ceph mgr.y for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:02:15.476 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:15 smithi171 ceph-mon[33189]: Deploying daemon mgr.y on smithi097 2024-04-25T17:02:15.476 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:15.139+0000 7fdc1a091200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:02:15.476 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:15.308+0000 7fdc1a091200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:02:15.476 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:15.477+0000 7fdc1a091200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:02:15.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 -- 172.21.15.96:0/2114591251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 msgr2=0x7f774c104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:15.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 --2- 172.21.15.96:0/2114591251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c104ef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7748009990 tx=0x7f7748009ca0 comp rx=0 tx=0).stop 2024-04-25T17:02:15.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 -- 172.21.15.96:0/2114591251 shutdown_connections 2024-04-25T17:02:15.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 --2- 172.21.15.96:0/2114591251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f774c107d90 0x7f774c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:15.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 --2- 172.21.15.96:0/2114591251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c104ef0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:15.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 --2- 172.21.15.96:0/2114591251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c105430 0x7f774c107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:15.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 -- 172.21.15.96:0/2114591251 >> 172.21.15.96:0/2114591251 conn(0x7f774c0fe740 msgr2=0x7f774c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:15.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 -- 172.21.15.96:0/2114591251 shutdown_connections 2024-04-25T17:02:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.560+0000 7f7753b08700 1 -- 172.21.15.96:0/2114591251 wait complete. 2024-04-25T17:02:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 Processor -- start 2024-04-25T17:02:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 -- start start 2024-04-25T17:02:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c105430 0x7f774c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f774c107d90 0x7f774c1a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f774c116260 con 0x7f774c102ad0 2024-04-25T17:02:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1160e0 con 0x7f774c105430 2024-04-25T17:02:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f7753b08700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1163e0 con 0x7f774c107d90 2024-04-25T17:02:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f77518a4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f77518a4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c1a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44956/0 (socket says 172.21.15.96:44956) 2024-04-25T17:02:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f77518a4700 1 -- 172.21.15.96:0/318119338 learned_addr learned my addr 172.21.15.96:0/318119338 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:02:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f77510a3700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c105430 0x7f774c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.561+0000 7f77520a5700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f774c107d90 0x7f774c1a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f77518a4700 1 -- 172.21.15.96:0/318119338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f774c107d90 msgr2=0x7f774c1a82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f77518a4700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f774c107d90 0x7f774c1a82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f77518a4700 1 -- 172.21.15.96:0/318119338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c105430 msgr2=0x7f774c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f77518a4700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c105430 0x7f774c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f77518a4700 1 -- 172.21.15.96:0/318119338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c1a8940 con 0x7f774c102ad0 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f77510a3700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c105430 0x7f774c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f77518a4700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c1a0de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7748003800 tx=0x7f77480057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748026070 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7748004000 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c1a8bd0 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774800fcd0 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.562+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c1a9080 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.563+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f7748029070 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.563+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774c051190 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.563+0000 7f7742ffd700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f773803d850 0x7f773803fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.563+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7748052360 con 0x7f774c102ad0 2024-04-25T17:02:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.563+0000 7f77510a3700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f773803d850 0x7f773803fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:15.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.564+0000 7f77510a3700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f773803d850 0x7f773803fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f774c1a1e90 tx=0x7f773c0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:15.569 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 podman[33974]: 2024-04-25 17:02:15.199676287 +0000 UTC m=+0.017876884 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:02:15.569 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 podman[33974]: 2024-04-25 17:02:15.323137151 +0000 UTC m=+0.141337743 container create fb87bae1086983a38364ad3451fac38709267693a779a65fe2a4e5ef475cefa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T17:02:15.570 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 podman[33974]: 2024-04-25 17:02:15.533748194 +0000 UTC m=+0.351948778 container init fb87bae1086983a38364ad3451fac38709267693a779a65fe2a4e5ef475cefa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T17:02:15.570 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 podman[33974]: 2024-04-25 17:02:15.536379447 +0000 UTC m=+0.354580039 container start fb87bae1086983a38364ad3451fac38709267693a779a65fe2a4e5ef475cefa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T17:02:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.569+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7748013070 con 0x7f774c102ad0 2024-04-25T17:02:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:15.783+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f774c063ad0 con 0x7f773803d850 2024-04-25T17:02:15.896 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 ceph-mgr[33993]: -- 172.21.15.97:0/3679169760 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55980d9005a0 con 0x55981268a000 2024-04-25T17:02:15.897 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 bash[33974]: fb87bae1086983a38364ad3451fac38709267693a779a65fe2a4e5ef475cefa2 2024-04-25T17:02:15.897 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 systemd[1]: Started Ceph mgr.y for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:02:15.897 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:15.711+0000 7f43f0c97200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:02:15.897 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:15.898+0000 7f43f0c97200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:02:16.028 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:15.764+0000 7fdc1a091200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:02:16.029 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:15.856+0000 7fdc1a091200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:02:16.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:16.002+0000 7f43f0c97200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:02:16.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:16.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:02:16.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:02:16.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:16.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:16.393 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:16.393 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.394 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.394 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.394 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.394 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:02:16.394 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:02:16.394 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:16.394 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:16.528 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:16.067+0000 7fdc1a091200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:16.267+0000 7fdc1a091200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:16.732 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:16.327+0000 7f43f0c97200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:02:16.732 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:16.429+0000 7f43f0c97200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:02:16.989 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:16.910+0000 7fdc1a091200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:02:17.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:17 smithi171 ceph-mon[33189]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:17.396 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:17 smithi096 ceph-mon[29131]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:17.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:17 smithi097 ceph-mon[32602]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:17.650 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:17.354+0000 7fdc1a091200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:02:17.651 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:17.453+0000 7fdc1a091200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:02:17.651 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:17.652+0000 7fdc1a091200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:02:17.982 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:17 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:17.667+0000 7f43f0c97200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:02:17.982 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:17 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:17.834+0000 7f43f0c97200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:02:18.291 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:18.004+0000 7f43f0c97200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:18.292 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:18.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:18.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.504 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:18.173+0000 7fdc1a091200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:02:18.504 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:18.264+0000 7fdc1a091200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:02:18.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:18.504 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:18.505 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:18.585 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:18.293+0000 7f43f0c97200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:02:18.586 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:18.385+0000 7f43f0c97200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:02:18.586 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:18.587+0000 7f43f0c97200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:02:18.778 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:18.506+0000 7fdc1a091200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:02:18.778 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:18.639+0000 7fdc1a091200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:02:18.778 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:18.733+0000 7fdc1a091200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:02:18.982 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:18.775+0000 7f43f0c97200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:02:19.150 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:18.849+0000 7fdc1a091200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:02:19.150 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:19.152+0000 7fdc1a091200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:02:19.151 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-mon[33189]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T17:02:19.151 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-mon[33189]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T17:02:19.151 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2679177750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0086d3f-f5b8-46e8-95db-03996e609f38"}]: dispatch 2024-04-25T17:02:19.151 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2679177750' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0086d3f-f5b8-46e8-95db-03996e609f38"}]': finished 2024-04-25T17:02:19.151 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-mon[33189]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T17:02:19.151 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:19.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:19 smithi096 ceph-mon[29131]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T17:02:19.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:19 smithi096 ceph-mon[29131]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T17:02:19.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:19 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2679177750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0086d3f-f5b8-46e8-95db-03996e609f38"}]: dispatch 2024-04-25T17:02:19.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:19 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2679177750' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0086d3f-f5b8-46e8-95db-03996e609f38"}]': finished 2024-04-25T17:02:19.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:19 smithi096 ceph-mon[29131]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T17:02:19.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:19.371 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-mon[32602]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T17:02:19.371 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-mon[32602]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T17:02:19.371 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2679177750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0086d3f-f5b8-46e8-95db-03996e609f38"}]: dispatch 2024-04-25T17:02:19.371 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2679177750' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0086d3f-f5b8-46e8-95db-03996e609f38"}]': finished 2024-04-25T17:02:19.371 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-mon[32602]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T17:02:19.371 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:19.732 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:19.372+0000 7f43f0c97200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:02:20.028 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:19.750+0000 7fdc1a091200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:02:20.028 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:02:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:02:19.965+0000 7fdc1a091200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:02:20.067 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:19.791+0000 7f43f0c97200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:02:20.067 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:19.886+0000 7f43f0c97200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:02:20.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:20.412+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 14) ==== 98518+0+0 (secure 0 0 0) 0x7f774800f460 con 0x7f774c102ad0 2024-04-25T17:02:20.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:20 smithi096 ceph-mon[29131]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:20.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:20 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1068122174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:02:20.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:20 smithi096 ceph-mon[29131]: Standby manager daemon z started 2024-04-25T17:02:20.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:20 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T17:02:20.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:20 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:02:20.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:20 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T17:02:20.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:20 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:02:20.482 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:20.088+0000 7f43f0c97200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:02:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-mon[32602]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1068122174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:02:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-mon[32602]: Standby manager daemon z started 2024-04-25T17:02:20.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T17:02:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:02:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T17:02:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:02:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:20 smithi171 ceph-mon[33189]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:20 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1068122174' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:02:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:20 smithi171 ceph-mon[33189]: Standby manager daemon z started 2024-04-25T17:02:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:20 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T17:02:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:20 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:02:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:20 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T17:02:20.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:20 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3244920747' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:02:20.938 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:20.609+0000 7f43f0c97200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:02:20.939 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:20.700+0000 7f43f0c97200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:02:21.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:20.940+0000 7f43f0c97200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:02:21.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:21.072+0000 7f43f0c97200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:02:21.232 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:21.165+0000 7f43f0c97200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:02:21.578 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:21 smithi097 ceph-mon[32602]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T17:02:21.578 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:21 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T17:02:21.578 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:21 smithi097 ceph-mon[32602]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:21.579 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:21.278+0000 7f43f0c97200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:02:21.579 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:21.580+0000 7f43f0c97200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:02:21.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:21 smithi096 ceph-mon[29131]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T17:02:21.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:21 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T17:02:21.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:21 smithi096 ceph-mon[29131]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:21.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:21 smithi171 ceph-mon[33189]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T17:02:21.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:21 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T17:02:21.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:21 smithi171 ceph-mon[33189]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:22.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:22.425+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f77480173f0 con 0x7f774c102ad0 2024-04-25T17:02:22.482 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:22.170+0000 7f43f0c97200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:02:22.482 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:02:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:02:22.383+0000 7f43f0c97200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:02:22.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:22 smithi097 ceph-mon[32602]: Standby manager daemon y started 2024-04-25T17:02:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:22 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T17:02:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:22 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:02:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:22 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T17:02:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:22 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:02:22.592 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:22 smithi096 ceph-mon[29131]: Standby manager daemon y started 2024-04-25T17:02:22.592 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:22 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T17:02:22.592 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:22 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:02:22.592 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:22 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T17:02:22.592 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:22 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:02:22.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:22 smithi171 ceph-mon[33189]: Standby manager daemon y started 2024-04-25T17:02:22.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:22 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T17:02:22.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:22 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:02:22.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:22 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T17:02:22.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:22 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/1392014172' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:02:23.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:23 smithi096 ceph-mon[29131]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T17:02:23.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:23 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T17:02:23.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:23 smithi096 ceph-mon[29131]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:23.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:23 smithi097 ceph-mon[32602]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T17:02:23.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:23 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T17:02:23.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:23 smithi097 ceph-mon[32602]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:23.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:23 smithi171 ceph-mon[33189]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T17:02:23.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:23 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T17:02:23.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:23 smithi171 ceph-mon[33189]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:25.432 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.433 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.433 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:25.433 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:25.433 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:25.433 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.433 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:25 smithi096 ceph-mon[29131]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:25.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:25.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:25 smithi097 ceph-mon[32602]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:25.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:25 smithi171 ceph-mon[33189]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:26.533 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T17:02:26.534 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:26.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T17:02:26.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:26.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T17:02:26.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:27.505 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:27 smithi096 ceph-mon[29131]: Deploying daemon osd.0 on smithi096 2024-04-25T17:02:27.505 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:27 smithi096 ceph-mon[29131]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:27.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:27 smithi097 ceph-mon[32602]: Deploying daemon osd.0 on smithi096 2024-04-25T17:02:27.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:27 smithi097 ceph-mon[32602]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:27.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:27 smithi171 ceph-mon[33189]: Deploying daemon osd.0 on smithi096 2024-04-25T17:02:27.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:27 smithi171 ceph-mon[33189]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:30 smithi096 ceph-mon[29131]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:30.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:30 smithi171 ceph-mon[33189]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:30.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:30 smithi097 ceph-mon[32602]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:32.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:32 smithi097 ceph-mon[32602]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:32.284 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:32.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:32.285 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:32.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:32 smithi096 ceph-mon[29131]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:32.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:32.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:32.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:32.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:32 smithi171 ceph-mon[33189]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:32.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:32.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:32.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:34.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:34 smithi096 ceph-mon[29131]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:34.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:34.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:34.067 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:34 smithi097 ceph-mon[32602]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:34.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:34 smithi171 ceph-mon[33189]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:34.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:34.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:34.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:34.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:35.347 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 0 on host 'smithi096' 2024-04-25T17:02:35.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.349+0000 7f7742ffd700 1 -- 172.21.15.96:0/318119338 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f774c063ad0 con 0x7f773803d850 2024-04-25T17:02:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f773803d850 msgr2=0x7f773803fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f773803d850 0x7f773803fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f774c1a1e90 tx=0x7f773c0073d0 comp rx=0 tx=0).stop 2024-04-25T17:02:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 msgr2=0x7f774c1a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c1a0de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7748003800 tx=0x7f77480057c0 comp rx=0 tx=0).stop 2024-04-25T17:02:35.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 shutdown_connections 2024-04-25T17:02:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f774c107d90 0x7f774c1a82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f773803d850 0x7f773803fd60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f774c102ad0 0x7f774c1a0de0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 --2- 172.21.15.96:0/318119338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f774c105430 0x7f774c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.351+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 >> 172.21.15.96:0/318119338 conn(0x7f774c0fe740 msgr2=0x7f774c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.352+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 shutdown_connections 2024-04-25T17:02:35.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:02:35.352+0000 7f7753b08700 1 -- 172.21.15.96:0/318119338 wait complete. 2024-04-25T17:02:35.947 DEBUG:teuthology.orchestra.run.smithi096:osd.0> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T17:02:35.949 INFO:tasks.cephadm:Deploying osd.1 on smithi097 with /dev/vg_nvme/lv_4... 2024-04-25T17:02:35.949 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T17:02:36.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:36 smithi096 ceph-mon[29131]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:36.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:36.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:36.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:36.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:36 smithi096 ceph-mon[29131]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T17:02:36.195 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:02:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:36 smithi097 ceph-mon[32602]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:36 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:36 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:36 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:36.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:36 smithi097 ceph-mon[32602]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T17:02:36.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:36 smithi171 ceph-mon[33189]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:36.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:36 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:36.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:36 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:36.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:36 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:36.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:36 smithi171 ceph-mon[33189]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T17:02:37.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:37 smithi096 ceph-mon[29131]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T17:02:37.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:37 smithi096 ceph-mon[29131]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T17:02:37.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:37 smithi096 ceph-mon[29131]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:02:37.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:37.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:37 smithi096 ceph-mon[29131]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:37.419 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:02:37 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0[43637]: 2024-04-25T17:02:37.324+0000 7fc3052eb700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T17:02:37.419 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:02:37 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0[43637]: 2024-04-25T17:02:37.328+0000 7fc3010a0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:02:37.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:37 smithi097 ceph-mon[32602]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T17:02:37.575 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:37 smithi097 ceph-mon[32602]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T17:02:37.575 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:37 smithi097 ceph-mon[32602]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:02:37.575 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:37.575 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:37 smithi097 ceph-mon[32602]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:37.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:37 smithi171 ceph-mon[33189]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T17:02:37.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:37 smithi171 ceph-mon[33189]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T17:02:37.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:37 smithi171 ceph-mon[33189]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:02:37.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:37.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:37 smithi171 ceph-mon[33189]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:38.130 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T17:02:38.130 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T17:02:38.130 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T17:02:38.130 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T17:02:38.130 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114226 s, 918 MB/s 2024-04-25T17:02:38.130 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T17:02:38.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:38 smithi096 ceph-mon[29131]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:02:38.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:38 smithi096 ceph-mon[29131]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T17:02:38.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:38.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:38.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:38 smithi097 ceph-mon[32602]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:02:38.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:38 smithi097 ceph-mon[32602]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T17:02:38.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:38.686 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:38.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-04-25T17:02:38.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:38 smithi171 ceph-mon[33189]: from='osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:02:38.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:38 smithi171 ceph-mon[33189]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T17:02:38.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:38.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:02:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:02:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:02:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] boot 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi096 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] boot 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi096 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:39.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: osd.0 [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] boot 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi096 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:02:39.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 -- 172.21.15.97:0/4051751342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0104210 msgr2=0x7fe8e0104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4051751342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0104210 0x7fe8e0104620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009990 tx=0x7fe8d0009ca0 comp rx=0 tx=0).stop 2024-04-25T17:02:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 -- 172.21.15.97:0/4051751342 shutdown_connections 2024-04-25T17:02:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4051751342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8e0105ce0 0x7fe8e010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4051751342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0104210 0x7fe8e0104620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4051751342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e0105340 0x7fe8e0105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:40.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 -- 172.21.15.97:0/4051751342 >> 172.21.15.97:0/4051751342 conn(0x7fe8e00ff870 msgr2=0x7fe8e0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:40.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 -- 172.21.15.97:0/4051751342 shutdown_connections 2024-04-25T17:02:40.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.544+0000 7fe8e5306700 1 -- 172.21.15.97:0/4051751342 wait complete. 2024-04-25T17:02:40.544 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:40 smithi097 ceph-mon[32602]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 Processor -- start 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 -- start start 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e0104210 0x7fe8e019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8e0105340 0x7fe8e019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0105ce0 0x7fe8e01a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010fcd0 con 0x7fe8e0104210 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010fb50 con 0x7fe8e0105ce0 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.545+0000 7fe8e5306700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e010fe50 con 0x7fe8e0105340 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0105ce0 0x7fe8e01a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:40.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0105ce0 0x7fe8e01a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42430/0 (socket says 172.21.15.97:42430) 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 -- 172.21.15.97:0/4207462280 learned_addr learned my addr 172.21.15.97:0/4207462280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8d67fc700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8e0105340 0x7fe8e019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8deffd700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e0104210 0x7fe8e019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 -- 172.21.15.97:0/4207462280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8e0105340 msgr2=0x7fe8e019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8e0105340 0x7fe8e019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 -- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e0104210 msgr2=0x7fe8e019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e0104210 0x7fe8e019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 -- 172.21.15.97:0/4207462280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e01a2320 con 0x7fe8e0105ce0 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8df7fe700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0105ce0 0x7fe8e01a1d30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8007d70 tx=0x7fe8d800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8dcff9700 1 -- 172.21.15.97:0/4207462280 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8016020 con 0x7fe8e0105ce0 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e01a2610 con 0x7fe8e0105ce0 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8dcff9700 1 -- 172.21.15.97:0/4207462280 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d800ac90 con 0x7fe8e0105ce0 2024-04-25T17:02:40.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8dcff9700 1 -- 172.21.15.97:0/4207462280 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80115e0 con 0x7fe8e0105ce0 2024-04-25T17:02:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.546+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01a2b80 con 0x7fe8e0105ce0 2024-04-25T17:02:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.547+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e0051190 con 0x7fe8e0105ce0 2024-04-25T17:02:40.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.549+0000 7fe8dcff9700 1 -- 172.21.15.97:0/4207462280 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7fe8d8019020 con 0x7fe8e0105ce0 2024-04-25T17:02:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.549+0000 7fe8dcff9700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe8cc0afaf0 0x7fe8cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:02:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.550+0000 7fe8dcff9700 1 -- 172.21.15.97:0/4207462280 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fe8d80dd5a0 con 0x7fe8e0105ce0 2024-04-25T17:02:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.550+0000 7fe8deffd700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe8cc0afaf0 0x7fe8cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:02:40.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.550+0000 7fe8deffd700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe8cc0afaf0 0x7fe8cc0b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:02:40.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.553+0000 7fe8dcff9700 1 -- 172.21.15.97:0/4207462280 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8d8020020 con 0x7fe8e0105ce0 2024-04-25T17:02:40.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:40 smithi096 ceph-mon[29131]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T17:02:40.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:40.758+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe8e0063ad0 con 0x7fe8cc0afaf0 2024-04-25T17:02:40.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:40 smithi171 ceph-mon[33189]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T17:02:41.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:41 smithi097 ceph-mon[32602]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:41.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:41 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:02:41.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:41 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:02:41.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:41 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:41.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:41 smithi097 ceph-mon[32602]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:41.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:41 smithi096 ceph-mon[29131]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:41.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:02:41.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:02:41.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:41.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:41 smithi096 ceph-mon[29131]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:41.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:41 smithi171 ceph-mon[33189]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:02:41.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:41 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:02:41.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:41 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:02:41.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:41 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:41.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:41 smithi171 ceph-mon[33189]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:44 smithi097 ceph-mon[32602]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:44 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/4187513559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]: dispatch 2024-04-25T17:02:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:44 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]: dispatch 2024-04-25T17:02:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:44 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]': finished 2024-04-25T17:02:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:44 smithi097 ceph-mon[32602]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T17:02:44.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:44 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:02:44.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:44 smithi096 ceph-mon[29131]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:44.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:44 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/4187513559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]: dispatch 2024-04-25T17:02:44.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:44 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]: dispatch 2024-04-25T17:02:44.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:44 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]': finished 2024-04-25T17:02:44.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:44 smithi096 ceph-mon[29131]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T17:02:44.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:44 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:02:44.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:44 smithi171 ceph-mon[33189]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:44.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:44 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/4187513559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]: dispatch 2024-04-25T17:02:44.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:44 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]: dispatch 2024-04-25T17:02:44.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:44 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e42e40f-3157-4e43-b9fc-689bacb722b3"}]': finished 2024-04-25T17:02:44.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:44 smithi171 ceph-mon[33189]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T17:02:44.533 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:44 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:02:45.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:45 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/1117551685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:02:45.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:45 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/1117551685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:02:45.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:45 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/1117551685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:02:46.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:46 smithi096 ceph-mon[29131]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:46.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:46 smithi097 ceph-mon[32602]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:46.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:46 smithi171 ceph-mon[33189]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:48.395 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:48 smithi097 ceph-mon[32602]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:48.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:48 smithi096 ceph-mon[29131]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:48 smithi171 ceph-mon[33189]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:50.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:50 smithi097 ceph-mon[32602]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:50.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:50 smithi096 ceph-mon[29131]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:50 smithi171 ceph-mon[33189]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:51.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T17:02:51.303 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:51.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T17:02:51.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:51.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T17:02:51.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:02:52.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:52 smithi097 ceph-mon[32602]: Deploying daemon osd.1 on smithi097 2024-04-25T17:02:52.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:52 smithi097 ceph-mon[32602]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:52 smithi096 ceph-mon[29131]: Deploying daemon osd.1 on smithi097 2024-04-25T17:02:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:52 smithi096 ceph-mon[29131]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:52.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:52 smithi171 ceph-mon[33189]: Deploying daemon osd.1 on smithi097 2024-04-25T17:02:52.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:52 smithi171 ceph-mon[33189]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:54.399 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:54 smithi097 ceph-mon[32602]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:54.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:54 smithi096 ceph-mon[29131]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:54.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:54 smithi171 ceph-mon[33189]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:56.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:56 smithi097 ceph-mon[32602]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:56.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:56.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:56 smithi096 ceph-mon[29131]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:56.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:56.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:56 smithi171 ceph-mon[33189]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:56.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:57.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:57.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:57.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:57.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:57.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:02:57.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:02:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:02:58 smithi097 ceph-mon[32602]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:58.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:02:58 smithi096 ceph-mon[29131]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:58.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:02:58 smithi171 ceph-mon[33189]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:02:59.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.474+0000 7fe8dcff9700 1 -- 172.21.15.97:0/4207462280 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe8e0063ad0 con 0x7fe8cc0afaf0 2024-04-25T17:02:59.473 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 1 on host 'smithi097' 2024-04-25T17:02:59.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.476+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe8cc0afaf0 msgr2=0x7fe8cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:59.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.476+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe8cc0afaf0 0x7fe8cc0b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d001a040 comp rx=0 tx=0).stop 2024-04-25T17:02:59.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.476+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0105ce0 msgr2=0x7fe8e01a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.476+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0105ce0 0x7fe8e01a1d30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8007d70 tx=0x7fe8d800a340 comp rx=0 tx=0).stop 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 shutdown_connections 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8e0105ce0 0x7fe8e01a1d30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe8cc0afaf0 0x7fe8cc0b2000 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8e0104210 0x7fe8e019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 --2- 172.21.15.97:0/4207462280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8e0105340 0x7fe8e019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 >> 172.21.15.97:0/4207462280 conn(0x7fe8e00ff870 msgr2=0x7fe8e0108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 shutdown_connections 2024-04-25T17:02:59.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:02:59.477+0000 7fe8e5306700 1 -- 172.21.15.97:0/4207462280 wait complete. 2024-04-25T17:03:00.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:00 smithi097 ceph-mon[32602]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:00.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:00 smithi097 ceph-mon[32602]: from='osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T17:03:00.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:00 smithi097 ceph-mon[32602]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T17:03:00.276 DEBUG:teuthology.orchestra.run.smithi097:osd.1> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T17:03:00.278 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2024-04-25T17:03:00.278 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T17:03:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:00 smithi096 ceph-mon[29131]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:00 smithi096 ceph-mon[29131]: from='osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T17:03:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:00 smithi096 ceph-mon[29131]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T17:03:00.521 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:03:00.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:00 smithi171 ceph-mon[33189]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:00.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:00 smithi171 ceph-mon[33189]: from='osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T17:03:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:00 smithi171 ceph-mon[33189]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T17:03:01.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:01 smithi097 ceph-mon[32602]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T17:03:01.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:01 smithi097 ceph-mon[32602]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T17:03:01.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:01 smithi097 ceph-mon[32602]: from='osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:03:01.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:01.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:01 smithi097 ceph-mon[32602]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:03:01.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:01 smithi097 ceph-mon[32602]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:01.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:01 smithi171 ceph-mon[33189]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T17:03:01.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:01 smithi171 ceph-mon[33189]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T17:03:01.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:01 smithi171 ceph-mon[33189]: from='osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:03:01.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:01.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:01 smithi171 ceph-mon[33189]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:03:01.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:01 smithi171 ceph-mon[33189]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:01.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:01 smithi096 ceph-mon[29131]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T17:03:01.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:01 smithi096 ceph-mon[29131]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T17:03:01.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:01 smithi096 ceph-mon[29131]: from='osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:03:01.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:01.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:01 smithi096 ceph-mon[29131]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:03:01.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:01 smithi096 ceph-mon[29131]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:02.200 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T17:03:02.200 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T17:03:02.200 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T17:03:02.200 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T17:03:02.200 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114637 s, 915 MB/s 2024-04-25T17:03:02.200 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T17:03:02.286 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1[37175]: 2024-04-25T17:03:02.012+0000 7fcc589a9700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T17:03:02.286 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1[37175]: 2024-04-25T17:03:02.016+0000 7fcc50756700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:03:02.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:02 smithi171 ceph-mon[33189]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:03:02.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:02 smithi171 ceph-mon[33189]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T17:03:02.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:02 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:02.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:02 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:02.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:02 smithi171 ceph-mon[33189]: from='osd.1 ' entity='osd.1' 2024-04-25T17:03:02.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-04-25T17:03:02.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:02 smithi097 ceph-mon[32602]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:03:02.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:02 smithi097 ceph-mon[32602]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T17:03:02.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:02 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:02.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:02 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:02.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:02 smithi097 ceph-mon[32602]: from='osd.1 ' entity='osd.1' 2024-04-25T17:03:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:02 smithi096 ceph-mon[29131]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:03:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:02 smithi096 ceph-mon[29131]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T17:03:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:02 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:02 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:02 smithi096 ceph-mon[29131]: from='osd.1 ' entity='osd.1' 2024-04-25T17:03:03.092 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] boot 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:03.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:03:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:03:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] boot 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: osd.1 [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] boot 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:03.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:04.572 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 -- 172.21.15.171:0/2713790661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 msgr2=0x7f527410c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:04.572 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 --2- 172.21.15.171:0/2713790661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f527410c3d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f526c009990 tx=0x7f526c009ca0 comp rx=0 tx=0).stop 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 -- 172.21.15.171:0/2713790661 shutdown_connections 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 --2- 172.21.15.171:0/2713790661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f527410c3d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 --2- 172.21.15.171:0/2713790661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f527406c1a0 0x7f5274106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 --2- 172.21.15.171:0/2713790661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5274107350 0x7f5274109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 -- 172.21.15.171:0/2713790661 >> 172.21.15.171:0/2713790661 conn(0x7f52740fe750 msgr2=0x7f5274100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 -- 172.21.15.171:0/2713790661 shutdown_connections 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.572+0000 7f52796bb700 1 -- 172.21.15.171:0/2713790661 wait complete. 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 Processor -- start 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 -- start start 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f527406c1a0 0x7f527419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5274107350 0x7f527419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:04.573 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f52741a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f527410fc60 con 0x7f527406c1a0 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f527410fae0 con 0x7f5274107350 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.573+0000 7f52796bb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f527410fde0 con 0x7f5274109ec0 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f52741a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f52741a1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:42616/0 (socket says 172.21.15.171:42616) 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 -- 172.21.15.171:0/3238322084 learned_addr learned my addr 172.21.15.171:0/3238322084 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52727fc700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5274107350 0x7f527419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f5272ffd700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f527406c1a0 0x7f527419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 -- 172.21.15.171:0/3238322084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5274107350 msgr2=0x7f527419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5274107350 0x7f527419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 -- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f527406c1a0 msgr2=0x7f527419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f527406c1a0 0x7f527419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:04.574 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 -- 172.21.15.171:0/3238322084 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52741a21c0 con 0x7f5274109ec0 2024-04-25T17:03:04.575 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52737fe700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f52741a1bd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f526c0059d0 tx=0x7f526c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:03:04.575 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f526bfff700 1 -- 172.21.15.171:0/3238322084 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c027070 con 0x7f5274109ec0 2024-04-25T17:03:04.575 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52741a2450 con 0x7f5274109ec0 2024-04-25T17:03:04.575 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f526bfff700 1 -- 172.21.15.171:0/3238322084 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c022470 con 0x7f5274109ec0 2024-04-25T17:03:04.575 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.574+0000 7f526bfff700 1 -- 172.21.15.171:0/3238322084 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c017650 con 0x7f5274109ec0 2024-04-25T17:03:04.576 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.575+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52741a2900 con 0x7f5274109ec0 2024-04-25T17:03:04.576 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.576+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5274051190 con 0x7f5274109ec0 2024-04-25T17:03:04.576 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.577+0000 7f526bfff700 1 -- 172.21.15.171:0/3238322084 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f526c02a070 con 0x7f5274109ec0 2024-04-25T17:03:04.576 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.578+0000 7f526bfff700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52600afaf0 0x7f52600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:04.576 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.578+0000 7f526bfff700 1 -- 172.21.15.171:0/3238322084 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f526c0de9e0 con 0x7f5274109ec0 2024-04-25T17:03:04.576 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.578+0000 7f5272ffd700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52600afaf0 0x7f52600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:04.577 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.579+0000 7f5272ffd700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52600afaf0 0x7f52600b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5264009380 tx=0x7f526400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:03:04.579 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.582+0000 7f526bfff700 1 -- 172.21.15.171:0/3238322084 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f526c031020 con 0x7f5274109ec0 2024-04-25T17:03:04.784 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:04.786+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5274063ad0 con 0x7f52600afaf0 2024-04-25T17:03:05.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:05 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi097 2024-04-25T17:03:05.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:05 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T17:03:05.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:05 smithi171 ceph-mon[33189]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T17:03:05.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:05.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:05.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:05 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi097 2024-04-25T17:03:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:05 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T17:03:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:05 smithi096 ceph-mon[29131]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T17:03:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:05 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi097 2024-04-25T17:03:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:05 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T17:03:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:05 smithi097 ceph-mon[32602]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T17:03:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:06.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:06 smithi171 ceph-mon[33189]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:06.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:06 smithi171 ceph-mon[33189]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:06.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:06 smithi096 ceph-mon[29131]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:06.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:06 smithi096 ceph-mon[29131]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:06.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:06 smithi097 ceph-mon[32602]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:06.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:06 smithi097 ceph-mon[32602]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:08.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:08 smithi171 ceph-mon[33189]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:08.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:08 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/1772637228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]: dispatch 2024-04-25T17:03:08.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:08 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]: dispatch 2024-04-25T17:03:08.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:08 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]': finished 2024-04-25T17:03:08.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:08 smithi171 ceph-mon[33189]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T17:03:08.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:08.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:08 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/3833698234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:08.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:08 smithi096 ceph-mon[29131]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:08.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:08 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/1772637228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]: dispatch 2024-04-25T17:03:08.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:08 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]: dispatch 2024-04-25T17:03:08.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:08 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]': finished 2024-04-25T17:03:08.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:08 smithi096 ceph-mon[29131]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T17:03:08.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:08.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:08 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/3833698234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:08.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:08 smithi097 ceph-mon[32602]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:08.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:08 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/1772637228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]: dispatch 2024-04-25T17:03:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:08 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]: dispatch 2024-04-25T17:03:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:08 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a1d306b-692d-446a-a5af-40c55c06522e"}]': finished 2024-04-25T17:03:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:08 smithi097 ceph-mon[32602]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T17:03:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:08 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/3833698234' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:10.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:10 smithi096 ceph-mon[29131]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:10.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:10 smithi097 ceph-mon[32602]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:10.528 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:10 smithi171 ceph-mon[33189]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:12.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:12 smithi171 ceph-mon[33189]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:12.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:12 smithi096 ceph-mon[29131]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:12.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:12 smithi097 ceph-mon[32602]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:14 smithi171 ceph-mon[33189]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:14.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:14 smithi096 ceph-mon[29131]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:14.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:14 smithi097 ceph-mon[32602]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:15.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T17:03:15.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:15.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T17:03:15.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T17:03:15.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:16.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:16 smithi171 ceph-mon[33189]: Deploying daemon osd.2 on smithi171 2024-04-25T17:03:16.322 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:16 smithi171 ceph-mon[33189]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:16.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:16 smithi096 ceph-mon[29131]: Deploying daemon osd.2 on smithi171 2024-04-25T17:03:16.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:16 smithi096 ceph-mon[29131]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:16.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:16 smithi097 ceph-mon[32602]: Deploying daemon osd.2 on smithi171 2024-04-25T17:03:16.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:16 smithi097 ceph-mon[32602]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:18.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:18 smithi171 ceph-mon[33189]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:18 smithi096 ceph-mon[29131]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:18.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:18 smithi097 ceph-mon[32602]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:20.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:20 smithi171 ceph-mon[33189]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:20.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:20 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:20.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:20 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:20.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:20 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:20.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:20 smithi096 ceph-mon[29131]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:20.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:20 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:20.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:20 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:20.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:20 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:20 smithi097 ceph-mon[32602]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:20 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:20 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:20.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:20 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:22.155 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:22 smithi171 ceph-mon[33189]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:22 smithi096 ceph-mon[29131]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:22.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:22 smithi097 ceph-mon[32602]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:22.746 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.748+0000 7f526bfff700 1 -- 172.21.15.171:0/3238322084 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5274063ad0 con 0x7f52600afaf0 2024-04-25T17:03:22.747 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-04-25T17:03:22.749 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52600afaf0 msgr2=0x7f52600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:22.749 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52600afaf0 0x7f52600b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5264009380 tx=0x7f526400b040 comp rx=0 tx=0).stop 2024-04-25T17:03:22.749 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 msgr2=0x7f52741a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:22.749 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f52741a1bd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f526c0059d0 tx=0x7f526c0047b0 comp rx=0 tx=0).stop 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 shutdown_connections 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5274109ec0 0x7f52741a1bd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52600afaf0 0x7f52600b2000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f527406c1a0 0x7f527419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 --2- 172.21.15.171:0/3238322084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5274107350 0x7f527419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 >> 172.21.15.171:0/3238322084 conn(0x7f52740fe750 msgr2=0x7f52741057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 shutdown_connections 2024-04-25T17:03:22.750 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:03:22.751+0000 7f52796bb700 1 -- 172.21.15.171:0/3238322084 wait complete. 2024-04-25T17:03:23.368 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T17:03:23.370 INFO:tasks.cephadm:Deploying osd.3 on smithi096 with /dev/vg_nvme/lv_3... 2024-04-25T17:03:23.371 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T17:03:23.584 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2[38352]: 2024-04-25T17:03:23.509+0000 7f433c4817c0 -1 osd.2 0 log_to_monitors true 2024-04-25T17:03:23.616 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:03:23.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-mon[33189]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:23.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-mon[33189]: from='osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T17:03:23.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:23 smithi171 ceph-mon[33189]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T17:03:23.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:23 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:23 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:23 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:23 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:23 smithi096 ceph-mon[29131]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:23.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:23 smithi096 ceph-mon[29131]: from='osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T17:03:23.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:23 smithi096 ceph-mon[29131]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T17:03:23.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:23 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:23 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:23 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:23 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:23 smithi097 ceph-mon[32602]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:23 smithi097 ceph-mon[32602]: from='osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T17:03:23.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:23 smithi097 ceph-mon[32602]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T17:03:25.011 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:24 smithi096 ceph-mon[29131]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T17:03:25.011 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:24 smithi096 ceph-mon[29131]: from='osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:03:25.011 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:24 smithi096 ceph-mon[29131]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T17:03:25.011 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:24 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:25.011 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:24 smithi096 ceph-mon[29131]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:03:25.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:24 smithi097 ceph-mon[32602]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T17:03:25.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:24 smithi097 ceph-mon[32602]: from='osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:03:25.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:24 smithi097 ceph-mon[32602]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T17:03:25.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:24 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:25.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:24 smithi097 ceph-mon[32602]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:03:25.279 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:24 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2[38352]: 2024-04-25T17:03:24.861+0000 7f4331ae6700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T17:03:25.279 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:24 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2[38352]: 2024-04-25T17:03:24.865+0000 7f432d89b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:03:25.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:24 smithi171 ceph-mon[33189]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T17:03:25.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:24 smithi171 ceph-mon[33189]: from='osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:03:25.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:24 smithi171 ceph-mon[33189]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T17:03:25.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:24 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:25.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:24 smithi171 ceph-mon[33189]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:03:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T17:03:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T17:03:25.691 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T17:03:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T17:03:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112475 s, 932 MB/s 2024-04-25T17:03:25.691 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 2024-04-25T17:03:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T17:03:25.991 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:25 smithi096 ceph-mon[29131]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:03:25.991 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:25 smithi096 ceph-mon[29131]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T17:03:25.991 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:25.991 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:25.991 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:25 smithi096 ceph-mon[29131]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:26.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:25 smithi171 ceph-mon[33189]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:03:26.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:25 smithi171 ceph-mon[33189]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T17:03:26.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:26.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:26.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:25 smithi171 ceph-mon[33189]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:26.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:25 smithi097 ceph-mon[32602]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:03:26.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:25 smithi097 ceph-mon[32602]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T17:03:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:26.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:25 smithi097 ceph-mon[32602]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T17:03:26.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-04-25T17:03:26.626 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] boot 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:27.001 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:27.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:27.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:27.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:27.002 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] boot 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:27.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:27.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:27.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:27.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:27.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: osd.2 [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] boot 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:27.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:27.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:27.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:27.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:27.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:27.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:27.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:28.015 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:27 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi171 2024-04-25T17:03:28.015 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:27 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-25T17:03:28.015 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:27 smithi096 ceph-mon[29131]: pgmap v80: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:28.016 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:27 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:03:28.016 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:27 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T17:03:28.016 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:27 smithi096 ceph-mon[29131]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T17:03:28.016 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:27 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:03:28.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:27 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi171 2024-04-25T17:03:28.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:27 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-25T17:03:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:27 smithi097 ceph-mon[32602]: pgmap v80: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:27 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:03:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:27 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T17:03:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:27 smithi097 ceph-mon[32602]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T17:03:28.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:27 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:03:28.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 -- 172.21.15.96:0/3628754344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 msgr2=0x7f97bc10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:28.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 --2- 172.21.15.96:0/3628754344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc10b460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f97b400b0a0 tx=0x7f97b400b3b0 comp rx=0 tx=0).stop 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 -- 172.21.15.96:0/3628754344 shutdown_connections 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 --2- 172.21.15.96:0/3628754344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc10b460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 --2- 172.21.15.96:0/3628754344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97bc1030e0 0x7f97bc1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 --2- 172.21.15.96:0/3628754344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97bc104210 0x7f97bc104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 -- 172.21.15.96:0/3628754344 >> 172.21.15.96:0/3628754344 conn(0x7f97bc0fe740 msgr2=0x7f97bc100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 -- 172.21.15.96:0/3628754344 shutdown_connections 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.251+0000 7f97c299c700 1 -- 172.21.15.96:0/3628754344 wait complete. 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 Processor -- start 2024-04-25T17:03:28.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 -- start start 2024-04-25T17:03:28.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97bc1030e0 0x7f97bc076380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:28.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97bc104210 0x7f97bc19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:28.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:28.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc10ebc0 con 0x7f97bc104bb0 2024-04-25T17:03:28.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc10ea40 con 0x7f97bc1030e0 2024-04-25T17:03:28.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c299c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc10ed40 con 0x7f97bc104210 2024-04-25T17:03:28.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c0f39700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c0f39700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc1a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41452/0 (socket says 172.21.15.96:41452) 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97c0f39700 1 -- 172.21.15.96:0/4194224329 learned_addr learned my addr 172.21.15.96:0/4194224329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97bb7fe700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97bc104210 0x7f97bc19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.252+0000 7f97bbfff700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97bc1030e0 0x7f97bc076380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c0f39700 1 -- 172.21.15.96:0/4194224329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97bc104210 msgr2=0x7f97bc19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c0f39700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97bc104210 0x7f97bc19f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c0f39700 1 -- 172.21.15.96:0/4194224329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97bc1030e0 msgr2=0x7f97bc076380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c0f39700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97bc1030e0 0x7f97bc076380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c0f39700 1 -- 172.21.15.96:0/4194224329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc1a2310 con 0x7f97bc104bb0 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c0f39700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc1a1ae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97b400b830 tx=0x7f97b4004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4021070 con 0x7f97bc104bb0 2024-04-25T17:03:28.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc1a25a0 con 0x7f97bc104bb0 2024-04-25T17:03:28.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b4007b80 con 0x7f97bc104bb0 2024-04-25T17:03:28.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4004730 con 0x7f97bc104bb0 2024-04-25T17:03:28.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.253+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc1a2bb0 con 0x7f97bc104bb0 2024-04-25T17:03:28.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.254+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc051190 con 0x7f97bc104bb0 2024-04-25T17:03:28.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.256+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f97b4003a00 con 0x7f97bc104bb0 2024-04-25T17:03:28.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.256+0000 7f97b97fa700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f97a40afaf0 0x7f97a40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:28.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.256+0000 7f97bbfff700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f97a40afaf0 0x7f97a40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:28.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.257+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f97b400f020 con 0x7f97bc104bb0 2024-04-25T17:03:28.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.257+0000 7f97bbfff700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f97a40afaf0 0x7f97a40b2000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97bc104110 tx=0x7f97ac009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:03:28.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.260+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97b402a020 con 0x7f97bc104bb0 2024-04-25T17:03:28.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:27 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi171 2024-04-25T17:03:28.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:27 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-25T17:03:28.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:27 smithi171 ceph-mon[33189]: pgmap v80: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:28.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:27 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:03:28.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:27 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T17:03:28.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:27 smithi171 ceph-mon[33189]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T17:03:28.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:27 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:03:28.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:28.476+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f97bc02e980 con 0x7f97a40afaf0 2024-04-25T17:03:28.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48239]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T17:03:28.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48239]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48239]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48239]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:28.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48242]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T17:03:28.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48242]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48242]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48242]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48166]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48166]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48166]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48169]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.938 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:03:28 smithi096 sudo[48169]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:28.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T17:03:28.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41000]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41000]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:28.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T17:03:28.983 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.984 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41003]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.984 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41003]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T17:03:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41006]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41006]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41009]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T17:03:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41009]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:28 smithi097 sudo[41009]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42406]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42406]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42406]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42406]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42409]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:29.029 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42409]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42412]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42412]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42412]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42412]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42415]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42415]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42415]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T17:03:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:28 smithi171 sudo[42415]: pam_unix(sudo:session): session closed for user root 2024-04-25T17:03:29.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:29.650+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f97b40a4d50 con 0x7f97bc104bb0 2024-04-25T17:03:29.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T17:03:29.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T17:03:29.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:29.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:29.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:29.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:29.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:29 smithi096 ceph-mon[29131]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:29 smithi097 ceph-mon[32602]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:29.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T17:03:29.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T17:03:29.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:29.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:29.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:29.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:29.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:29.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:03:29.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:03:29.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T17:03:29.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:29 smithi171 ceph-mon[33189]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:30.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:30 smithi096 ceph-mon[29131]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T17:03:30.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:30 smithi096 ceph-mon[29131]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T17:03:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:30 smithi097 ceph-mon[32602]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T17:03:30.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:30 smithi097 ceph-mon[32602]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T17:03:30.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:30 smithi171 ceph-mon[33189]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T17:03:30.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:30 smithi171 ceph-mon[33189]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T17:03:31.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:31 smithi096 ceph-mon[29131]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:31.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:31 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3536910212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abdaf538-404e-4714-b1a2-dcc2c22f275b"}]: dispatch 2024-04-25T17:03:31.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:31 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3536910212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abdaf538-404e-4714-b1a2-dcc2c22f275b"}]': finished 2024-04-25T17:03:31.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:31 smithi096 ceph-mon[29131]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T17:03:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:31 smithi097 ceph-mon[32602]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:31 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3536910212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abdaf538-404e-4714-b1a2-dcc2c22f275b"}]: dispatch 2024-04-25T17:03:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:31 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3536910212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abdaf538-404e-4714-b1a2-dcc2c22f275b"}]': finished 2024-04-25T17:03:31.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:31 smithi097 ceph-mon[32602]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T17:03:31.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:31.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:31 smithi171 ceph-mon[33189]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:31.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:31 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3536910212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abdaf538-404e-4714-b1a2-dcc2c22f275b"}]: dispatch 2024-04-25T17:03:31.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:31 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3536910212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abdaf538-404e-4714-b1a2-dcc2c22f275b"}]': finished 2024-04-25T17:03:31.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:31 smithi171 ceph-mon[33189]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T17:03:31.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:32.680 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:32 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/880515172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:32.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:32 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/880515172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:32.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:32 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/880515172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:33.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:33 smithi096 ceph-mon[29131]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:33.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:33 smithi097 ceph-mon[32602]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:33.778 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:33 smithi171 ceph-mon[33189]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:36.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:36 smithi096 ceph-mon[29131]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 174 KiB/s wr, 11 op/s 2024-04-25T17:03:36.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:36 smithi097 ceph-mon[32602]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 174 KiB/s wr, 11 op/s 2024-04-25T17:03:36.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:36 smithi171 ceph-mon[33189]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 174 KiB/s wr, 11 op/s 2024-04-25T17:03:38.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:38 smithi096 ceph-mon[29131]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 130 KiB/s wr, 8 op/s 2024-04-25T17:03:38.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:38 smithi097 ceph-mon[32602]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 130 KiB/s wr, 8 op/s 2024-04-25T17:03:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:38 smithi171 ceph-mon[33189]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 130 KiB/s wr, 8 op/s 2024-04-25T17:03:39.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T17:03:39.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:39.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T17:03:39.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:39.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T17:03:39.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:40 smithi096 ceph-mon[29131]: Deploying daemon osd.3 on smithi096 2024-04-25T17:03:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:40 smithi096 ceph-mon[29131]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-04-25T17:03:40.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:40 smithi097 ceph-mon[32602]: Deploying daemon osd.3 on smithi096 2024-04-25T17:03:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:40 smithi097 ceph-mon[32602]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-04-25T17:03:40.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:40 smithi171 ceph-mon[33189]: Deploying daemon osd.3 on smithi096 2024-04-25T17:03:40.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:40 smithi171 ceph-mon[33189]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-04-25T17:03:42.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:42 smithi096 ceph-mon[29131]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 104 KiB/s wr, 6 op/s 2024-04-25T17:03:42.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:42 smithi097 ceph-mon[32602]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 104 KiB/s wr, 6 op/s 2024-04-25T17:03:42.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:42 smithi171 ceph-mon[33189]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 104 KiB/s wr, 6 op/s 2024-04-25T17:03:44.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:44 smithi097 ceph-mon[32602]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 89 KiB/s wr, 5 op/s 2024-04-25T17:03:44.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:44 smithi171 ceph-mon[33189]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 89 KiB/s wr, 5 op/s 2024-04-25T17:03:44.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:44 smithi096 ceph-mon[29131]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 89 KiB/s wr, 5 op/s 2024-04-25T17:03:45.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:45.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:45.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:45.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:45 smithi096 ceph-mon[29131]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 87 KiB/s wr, 5 op/s 2024-04-25T17:03:45.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:45.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:45.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:45.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:45 smithi097 ceph-mon[32602]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 87 KiB/s wr, 5 op/s 2024-04-25T17:03:45.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:45.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:45.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:45.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:45 smithi171 ceph-mon[33189]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 87 KiB/s wr, 5 op/s 2024-04-25T17:03:48.146 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:48 smithi096 ceph-mon[29131]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:48.146 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.146 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.146 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:48.146 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:48.146 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.233+0000 7f97b97fa700 1 -- 172.21.15.96:0/4194224329 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f97bc02e980 con 0x7f97a40afaf0 2024-04-25T17:03:48.232 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 3 on host 'smithi096' 2024-04-25T17:03:48.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.235+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f97a40afaf0 msgr2=0x7f97a40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.235+0000 7f97c299c700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f97a40afaf0 0x7f97a40b2000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97bc104110 tx=0x7f97ac009190 comp rx=0 tx=0).stop 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.235+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 msgr2=0x7f97bc1a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.235+0000 7f97c299c700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc1a1ae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97b400b830 tx=0x7f97b4004dc0 comp rx=0 tx=0).stop 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 shutdown_connections 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f97a40afaf0 0x7f97a40b2000 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc104bb0 0x7f97bc1a1ae0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97bc1030e0 0x7f97bc076380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 --2- 172.21.15.96:0/4194224329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97bc104210 0x7f97bc19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 >> 172.21.15.96:0/4194224329 conn(0x7f97bc0fe740 msgr2=0x7f97bc107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 shutdown_connections 2024-04-25T17:03:48.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:03:48.236+0000 7f97c299c700 1 -- 172.21.15.96:0/4194224329 wait complete. 2024-04-25T17:03:48.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:48 smithi097 ceph-mon[32602]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:48 smithi171 ceph-mon[33189]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:48.925 DEBUG:teuthology.orchestra.run.smithi096:osd.3> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T17:03:48.927 INFO:tasks.cephadm:Deploying osd.4 on smithi097 with /dev/vg_nvme/lv_3... 2024-04-25T17:03:48.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T17:03:49.172 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:03:49.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:49.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:49 smithi097 ceph-mon[32602]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T17:03:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:49 smithi097 ceph-mon[32602]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:49.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:49.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:49.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:49.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:49 smithi096 ceph-mon[29131]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T17:03:49.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:49 smithi096 ceph-mon[29131]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:03:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:49 smithi171 ceph-mon[33189]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T17:03:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:49 smithi171 ceph-mon[33189]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:50 smithi097 ceph-mon[32602]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T17:03:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:50 smithi097 ceph-mon[32602]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T17:03:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:50 smithi097 ceph-mon[32602]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:03:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:50 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:50 smithi171 ceph-mon[33189]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T17:03:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:50 smithi171 ceph-mon[33189]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T17:03:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:50 smithi171 ceph-mon[33189]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:03:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:50 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:50.537 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:50 smithi096 ceph-mon[29131]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T17:03:50.537 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:50 smithi096 ceph-mon[29131]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T17:03:50.537 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:50 smithi096 ceph-mon[29131]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:03:50.538 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:50 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:50.538 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:03:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:03:50.199+0000 7f92ba049700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T17:03:50.538 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:03:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:03:50.205+0000 7f92b5dfe700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:03:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T17:03:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T17:03:51.085 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T17:03:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T17:03:51.085 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011317 s, 927 MB/s 2024-04-25T17:03:51.085 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T17:03:51.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:51 smithi097 ceph-mon[32602]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:03:51.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:51 smithi097 ceph-mon[32602]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T17:03:51.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:51.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:51.389 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:51 smithi097 ceph-mon[32602]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:51.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:51 smithi171 ceph-mon[33189]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:03:51.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:51 smithi171 ceph-mon[33189]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T17:03:51.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:51.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:51.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:51 smithi171 ceph-mon[33189]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:51.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:51 smithi096 ceph-mon[29131]: from='osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:03:51.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:51 smithi096 ceph-mon[29131]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T17:03:51.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:51.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:51.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:51 smithi096 ceph-mon[29131]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T17:03:51.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-04-25T17:03:52.006 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:03:52.391 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:03:52.391 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:03:52.391 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:52.392 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] boot 2024-04-25T17:03:52.392 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T17:03:52.392 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:52.392 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.392 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.392 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:52.393 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:52.393 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.393 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:52.393 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:52.393 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:03:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:03:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] boot 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: osd.3 [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] boot 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:03:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:03:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:03:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:53 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi096 2024-04-25T17:03:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:53 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T17:03:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:53 smithi097 ceph-mon[32602]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T17:03:53.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:53 smithi097 ceph-mon[32602]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:03:53.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:53 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi096 2024-04-25T17:03:53.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:53 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T17:03:53.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:53 smithi171 ceph-mon[33189]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T17:03:53.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:53 smithi171 ceph-mon[33189]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:03:53.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:53 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi096 2024-04-25T17:03:53.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:53 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T17:03:53.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:53 smithi096 ceph-mon[29131]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T17:03:53.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:53 smithi096 ceph-mon[29131]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 -- 172.21.15.97:0/89543248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 msgr2=0x7f1108109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 --2- 172.21.15.97:0/89543248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f1108109830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f10fc009990 tx=0x7f10fc009ca0 comp rx=0 tx=0).stop 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 -- 172.21.15.97:0/89543248 shutdown_connections 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 --2- 172.21.15.97:0/89543248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1108109ec0 0x7f110810c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 --2- 172.21.15.97:0/89543248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f110806c1a0 0x7f1108106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 --2- 172.21.15.97:0/89543248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f1108109830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 -- 172.21.15.97:0/89543248 >> 172.21.15.97:0/89543248 conn(0x7f11080fe750 msgr2=0x7f1108100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 -- 172.21.15.97:0/89543248 shutdown_connections 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.730+0000 7f110db8a700 1 -- 172.21.15.97:0/89543248 wait complete. 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 Processor -- start 2024-04-25T17:03:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 -- start start 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110806c1a0 0x7f110819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f110819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1108109ec0 0x7f11081a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f110810fcb0 con 0x7f1108109ec0 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f110810fb30 con 0x7f1108107350 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f110db8a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f110810fe30 con 0x7f110806c1a0 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.731+0000 7f1106ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f110819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f110819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43454/0 (socket says 172.21.15.97:43454) 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 -- 172.21.15.97:0/2050807949 learned_addr learned my addr 172.21.15.97:0/2050807949 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f11077fe700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110806c1a0 0x7f110819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1107fff700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1108109ec0 0x7f11081a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 -- 172.21.15.97:0/2050807949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110806c1a0 msgr2=0x7f110819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110806c1a0 0x7f110819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 -- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1108109ec0 msgr2=0x7f11081a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1108109ec0 0x7f11081a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 -- 172.21.15.97:0/2050807949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11081a2330 con 0x7f1108107350 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f11077fe700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110806c1a0 0x7f110819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1106ffd700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f110819ad10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098e0 tx=0x7f10fc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1104ff9700 1 -- 172.21.15.97:0/2050807949 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc026070 con 0x7f1108107350 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1104ff9700 1 -- 172.21.15.97:0/2050807949 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10fc00fbc0 con 0x7f1108107350 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081a25c0 con 0x7f1108107350 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f1104ff9700 1 -- 172.21.15.97:0/2050807949 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc017650 con 0x7f1108107350 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.732+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081a2a90 con 0x7f1108107350 2024-04-25T17:03:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.733+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1108051190 con 0x7f1108107350 2024-04-25T17:03:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.735+0000 7f1104ff9700 1 -- 172.21.15.97:0/2050807949 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f10fc029070 con 0x7f1108107350 2024-04-25T17:03:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.736+0000 7f1104ff9700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f10f00afb80 0x7f10f00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:03:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.736+0000 7f11077fe700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f10f00afb80 0x7f10f00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:03:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.736+0000 7f1104ff9700 1 -- 172.21.15.97:0/2050807949 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f10fc0ded80 con 0x7f1108107350 2024-04-25T17:03:53.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.736+0000 7f11077fe700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f10f00afb80 0x7f10f00b2090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009be0 tx=0x7f10f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:03:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.740+0000 7f1104ff9700 1 -- 172.21.15.97:0/2050807949 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10fc030080 con 0x7f1108107350 2024-04-25T17:03:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:03:53.944+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1108063b90 con 0x7f10f00afb80 2024-04-25T17:03:54.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:54 smithi097 ceph-mon[32602]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T17:03:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:54 smithi097 ceph-mon[32602]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:54 smithi171 ceph-mon[33189]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T17:03:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:54 smithi171 ceph-mon[33189]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:54.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:54 smithi096 ceph-mon[29131]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T17:03:54.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:54 smithi096 ceph-mon[29131]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:03:54.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:03:54.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:03:54.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:03:55.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:55 smithi097 ceph-mon[32602]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-25T17:03:55.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:55 smithi171 ceph-mon[33189]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-25T17:03:55.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:55 smithi096 ceph-mon[29131]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-25T17:03:57.034 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:56 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/107015218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]: dispatch 2024-04-25T17:03:57.034 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:56 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]: dispatch 2024-04-25T17:03:57.034 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:56 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]': finished 2024-04-25T17:03:57.034 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:56 smithi171 ceph-mon[33189]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T17:03:57.034 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:03:57.082 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:56 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/107015218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]: dispatch 2024-04-25T17:03:57.082 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:56 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]: dispatch 2024-04-25T17:03:57.083 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:56 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]': finished 2024-04-25T17:03:57.083 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:56 smithi097 ceph-mon[32602]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T17:03:57.083 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:03:57.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:56 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/107015218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]: dispatch 2024-04-25T17:03:57.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:56 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]: dispatch 2024-04-25T17:03:57.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:56 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27140752-dc84-4eb3-a875-5637f11a800a"}]': finished 2024-04-25T17:03:57.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:56 smithi096 ceph-mon[29131]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T17:03:57.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:03:58.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:57 smithi171 ceph-mon[33189]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T17:03:58.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:03:57 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/2343519695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:58.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:57 smithi096 ceph-mon[29131]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T17:03:58.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:03:57 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/2343519695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:03:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:57 smithi097 ceph-mon[32602]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T17:03:58.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:03:57 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/2343519695' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:04:00.393 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:00 smithi097 ceph-mon[32602]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T17:04:00.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:00 smithi096 ceph-mon[29131]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T17:04:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:00 smithi171 ceph-mon[33189]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T17:04:02.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:02 smithi097 ceph-mon[32602]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:04:02.331 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:02 smithi171 ceph-mon[33189]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:04:02.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:02 smithi096 ceph-mon[29131]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:04:04.156 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:04 smithi097 ceph-mon[32602]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T17:04:04.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:04 smithi096 ceph-mon[29131]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T17:04:04.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:04 smithi171 ceph-mon[33189]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-25T17:04:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T17:04:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:05.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T17:04:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T17:04:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:06.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:06 smithi097 ceph-mon[32602]: Deploying daemon osd.4 on smithi097 2024-04-25T17:04:06.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:06 smithi097 ceph-mon[32602]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:06.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:06 smithi096 ceph-mon[29131]: Deploying daemon osd.4 on smithi097 2024-04-25T17:04:06.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:06 smithi096 ceph-mon[29131]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:06.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:06 smithi171 ceph-mon[33189]: Deploying daemon osd.4 on smithi097 2024-04-25T17:04:06.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:06 smithi171 ceph-mon[33189]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:08 smithi096 ceph-mon[29131]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:08.214 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:08 smithi097 ceph-mon[32602]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:08 smithi171 ceph-mon[33189]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:10.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:10 smithi096 ceph-mon[29131]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:10.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:10.468 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:10 smithi097 ceph-mon[32602]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:10.469 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:10 smithi171 ceph-mon[33189]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:11.379 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:11.379 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:11 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:11 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:11.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:11.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:11 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:12.367 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:12 smithi097 ceph-mon[32602]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:12.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:12 smithi096 ceph-mon[29131]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:12.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:12 smithi171 ceph-mon[33189]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:13.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.860+0000 7f1104ff9700 1 -- 172.21.15.97:0/2050807949 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1108063b90 con 0x7f10f00afb80 2024-04-25T17:04:13.860 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 4 on host 'smithi097' 2024-04-25T17:04:13.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f10f00afb80 msgr2=0x7f10f00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f10f00afb80 0x7f10f00b2090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009be0 tx=0x7f10f8009260 comp rx=0 tx=0).stop 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 msgr2=0x7f110819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f110819ad10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098e0 tx=0x7f10fc0047b0 comp rx=0 tx=0).stop 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 shutdown_connections 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f10f00afb80 0x7f10f00b2090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1108109ec0 0x7f11081a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110806c1a0 0x7f110819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 --2- 172.21.15.97:0/2050807949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107350 0x7f110819ad10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 >> 172.21.15.97:0/2050807949 conn(0x7f11080fe750 msgr2=0x7f11081057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 shutdown_connections 2024-04-25T17:04:13.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:04:13.863+0000 7f110db8a700 1 -- 172.21.15.97:0/2050807949 wait complete. 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:13 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:14.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:14.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:14.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:14.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:13 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:14.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:13 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:14.581 DEBUG:teuthology.orchestra.run.smithi097:osd.4> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T17:04:14.583 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2024-04-25T17:04:14.583 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T17:04:14.827 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:04:15.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:14 smithi096 ceph-mon[29131]: from='osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T17:04:15.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:14 smithi096 ceph-mon[29131]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T17:04:15.204 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:14 smithi171 ceph-mon[33189]: from='osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T17:04:15.204 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:14 smithi171 ceph-mon[33189]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T17:04:15.214 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:14 smithi097 ceph-mon[32602]: from='osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T17:04:15.214 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:14 smithi097 ceph-mon[32602]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T17:04:16.049 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:15 smithi097 ceph-mon[32602]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T17:04:16.050 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:15 smithi097 ceph-mon[32602]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T17:04:16.050 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:16.050 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:15 smithi097 ceph-mon[32602]: from='osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:04:16.050 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:15 smithi097 ceph-mon[32602]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:04:16.050 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:15 smithi097 ceph-mon[32602]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:16.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:15 smithi096 ceph-mon[29131]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T17:04:16.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:15 smithi096 ceph-mon[29131]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T17:04:16.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:16.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:15 smithi096 ceph-mon[29131]: from='osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:04:16.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:15 smithi096 ceph-mon[29131]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:04:16.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:15 smithi096 ceph-mon[29131]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:16.212 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:15 smithi171 ceph-mon[33189]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T17:04:16.212 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:15 smithi171 ceph-mon[33189]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T17:04:16.212 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:16.212 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:15 smithi171 ceph-mon[33189]: from='osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:04:16.212 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:15 smithi171 ceph-mon[33189]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:04:16.212 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:15 smithi171 ceph-mon[33189]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:16.733 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:04:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4[43840]: 2024-04-25T17:04:16.357+0000 7f49aa8f2700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T17:04:16.733 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:04:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4[43840]: 2024-04-25T17:04:16.363+0000 7f49a269f700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:04:16.786 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T17:04:16.786 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T17:04:16.786 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T17:04:16.786 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T17:04:16.786 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114806 s, 913 MB/s 2024-04-25T17:04:16.786 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T17:04:17.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:16 smithi171 ceph-mon[33189]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:04:17.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:16 smithi171 ceph-mon[33189]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T17:04:17.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:17.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:16 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:17.097 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:16 smithi171 ceph-mon[33189]: from='osd.4 ' entity='osd.4' 2024-04-25T17:04:17.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:16 smithi096 ceph-mon[29131]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:04:17.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:16 smithi096 ceph-mon[29131]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T17:04:17.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:17.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:16 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:17.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:16 smithi096 ceph-mon[29131]: from='osd.4 ' entity='osd.4' 2024-04-25T17:04:17.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:16 smithi097 ceph-mon[32602]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:04:17.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:16 smithi097 ceph-mon[32602]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T17:04:17.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:17.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:16 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:17.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:16 smithi097 ceph-mon[32602]: from='osd.4 ' entity='osd.4' 2024-04-25T17:04:17.416 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-04-25T17:04:17.664 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:04:18.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:04:18.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:04:18.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:18.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:18.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] boot 2024-04-25T17:04:18.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T17:04:18.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:18.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:17 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:04:18.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] boot 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:17 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: osd.4 [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] boot 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:17 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:19.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:18 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi097 2024-04-25T17:04:19.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:18 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T17:04:19.198 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:18 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi097 2024-04-25T17:04:19.198 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:18 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T17:04:19.211 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.212+0000 7f4d6a384700 1 -- 172.21.15.171:0/2069148280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64107660 msgr2=0x7f4d64109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:19.211 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.212+0000 7f4d6a384700 1 --2- 172.21.15.171:0/2069148280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64107660 0x7f4d64109a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009990 tx=0x7f4d58009ca0 comp rx=0 tx=0).stop 2024-04-25T17:04:19.211 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.212+0000 7f4d6a384700 1 -- 172.21.15.171:0/2069148280 shutdown_connections 2024-04-25T17:04:19.211 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.212+0000 7f4d6a384700 1 --2- 172.21.15.171:0/2069148280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d64109fc0 0x7f4d6410c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:19.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.213+0000 7f4d6a384700 1 --2- 172.21.15.171:0/2069148280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d64104d00 0x7f4d64107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:19.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.213+0000 7f4d6a384700 1 --2- 172.21.15.171:0/2069148280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64107660 0x7f4d64109a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:19.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.213+0000 7f4d6a384700 1 -- 172.21.15.171:0/2069148280 >> 172.21.15.171:0/2069148280 conn(0x7f4d640fe740 msgr2=0x7f4d64100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:04:19.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.213+0000 7f4d6a384700 1 -- 172.21.15.171:0/2069148280 shutdown_connections 2024-04-25T17:04:19.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.213+0000 7f4d6a384700 1 -- 172.21.15.171:0/2069148280 wait complete. 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.213+0000 7f4d6a384700 1 Processor -- start 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.213+0000 7f4d6a384700 1 -- start start 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d6a384700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64104d00 0x7f4d64104560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d6a384700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d64107660 0x7f4d64102c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d6a384700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d64109fc0 0x7f4d64103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d6a384700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410fc50 con 0x7f4d64107660 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d6a384700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410fad0 con 0x7f4d64109fc0 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d6a384700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6410fdd0 con 0x7f4d64104d00 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64104d00 0x7f4d64104560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:19.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64104d00 0x7f4d64104560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:44660/0 (socket says 172.21.15.171:44660) 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 -- 172.21.15.171:0/1099260076 learned_addr learned my addr 172.21.15.171:0/1099260076 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d68921700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d64109fc0 0x7f4d64103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d637fe700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d64107660 0x7f4d64102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 -- 172.21.15.171:0/1099260076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d64109fc0 msgr2=0x7f4d64103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d64109fc0 0x7f4d64103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 -- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d64107660 msgr2=0x7f4d64102c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d64107660 0x7f4d64102c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 -- 172.21.15.171:0/1099260076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d641038b0 con 0x7f4d64104d00 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d63fff700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64104d00 0x7f4d64104560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400d980 tx=0x7f4d5400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d617fa700 1 -- 172.21.15.171:0/1099260076 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54017070 con 0x7f4d64104d00 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.214+0000 7f4d617fa700 1 -- 172.21.15.171:0/1099260076 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d54009de0 con 0x7f4d64104d00 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.215+0000 7f4d617fa700 1 -- 172.21.15.171:0/1099260076 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400f3c0 con 0x7f4d64104d00 2024-04-25T17:04:19.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.215+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d641a4890 con 0x7f4d64104d00 2024-04-25T17:04:19.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.215+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d641a4d20 con 0x7f4d64104d00 2024-04-25T17:04:19.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.216+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d64051190 con 0x7f4d64104d00 2024-04-25T17:04:19.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.217+0000 7f4d617fa700 1 -- 172.21.15.171:0/1099260076 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4d54009930 con 0x7f4d64104d00 2024-04-25T17:04:19.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.217+0000 7f4d617fa700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d4c0afc00 0x7f4d4c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:19.218 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.218+0000 7f4d617fa700 1 -- 172.21.15.171:0/1099260076 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f4d540dd660 con 0x7f4d64104d00 2024-04-25T17:04:19.218 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.218+0000 7f4d637fe700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d4c0afc00 0x7f4d4c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:19.218 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.218+0000 7f4d637fe700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d4c0afc00 0x7f4d4c0b2110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d580038a0 tx=0x7f4d5801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:04:19.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.222+0000 7f4d617fa700 1 -- 172.21.15.171:0/1099260076 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d540a72b0 con 0x7f4d64104d00 2024-04-25T17:04:19.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:18 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi097 2024-04-25T17:04:19.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:18 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T17:04:19.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:19.427+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4d64063ad0 con 0x7f4d4c0afc00 2024-04-25T17:04:20.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:19 smithi096 ceph-mon[29131]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T17:04:20.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:19 smithi096 ceph-mon[29131]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:20.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:04:20.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:04:20.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:20.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:19 smithi097 ceph-mon[32602]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T17:04:20.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:19 smithi097 ceph-mon[32602]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:20.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:19 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:04:20.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:19 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:04:20.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:19 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:20.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:19 smithi171 ceph-mon[33189]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T17:04:20.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:19 smithi171 ceph-mon[33189]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:20.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:19 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:04:20.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:19 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:04:20.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:19 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:21.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:20 smithi096 ceph-mon[29131]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:04:21.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:20 smithi097 ceph-mon[32602]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:04:21.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:20 smithi171 ceph-mon[33189]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:04:22.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:21 smithi097 ceph-mon[32602]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:22.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:21 smithi171 ceph-mon[33189]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:21 smithi096 ceph-mon[29131]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:23.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:22 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/688448615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]: dispatch 2024-04-25T17:04:23.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:22 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]: dispatch 2024-04-25T17:04:23.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:22 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]': finished 2024-04-25T17:04:23.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:22 smithi097 ceph-mon[32602]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T17:04:23.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:23.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:22 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/2433268090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:04:23.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:22 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/688448615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]: dispatch 2024-04-25T17:04:23.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:22 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]: dispatch 2024-04-25T17:04:23.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:22 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]': finished 2024-04-25T17:04:23.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:22 smithi171 ceph-mon[33189]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T17:04:23.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:23.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:22 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/2433268090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:04:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:22 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/688448615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]: dispatch 2024-04-25T17:04:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:22 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]: dispatch 2024-04-25T17:04:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:22 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d623b6a2-12d2-4564-a836-e726059745f9"}]': finished 2024-04-25T17:04:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:22 smithi096 ceph-mon[29131]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T17:04:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:23.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:22 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/2433268090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:04:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:23 smithi097 ceph-mon[32602]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:24.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:23 smithi171 ceph-mon[33189]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:24.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:23 smithi096 ceph-mon[29131]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:26.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:26 smithi171 ceph-mon[33189]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:26.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:26 smithi096 ceph-mon[29131]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:26.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:26 smithi097 ceph-mon[32602]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:28.383 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:28 smithi171 ceph-mon[33189]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:28.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:28 smithi096 ceph-mon[29131]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:28 smithi097 ceph-mon[32602]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:30.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:30 smithi171 ceph-mon[33189]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:30.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:30 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T17:04:30.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:30 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:30.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:30 smithi096 ceph-mon[29131]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:30.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:30 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T17:04:30.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:30 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:30.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:30 smithi097 ceph-mon[32602]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:30 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T17:04:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:30 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:31.275 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:31 smithi171 ceph-mon[33189]: Deploying daemon osd.5 on smithi171 2024-04-25T17:04:31.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:31 smithi096 ceph-mon[29131]: Deploying daemon osd.5 on smithi171 2024-04-25T17:04:31.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:31 smithi097 ceph-mon[32602]: Deploying daemon osd.5 on smithi171 2024-04-25T17:04:32.384 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:32 smithi097 ceph-mon[32602]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:32.406 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:32 smithi171 ceph-mon[33189]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:32 smithi096 ceph-mon[29131]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:34.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:34 smithi171 ceph-mon[33189]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:34.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:34 smithi096 ceph-mon[29131]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:34.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:34 smithi097 ceph-mon[32602]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:36.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:36 smithi171 ceph-mon[33189]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:36.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:36 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:36.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:36 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:36.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:36 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:36.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:36 smithi096 ceph-mon[29131]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:36.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:36.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:36.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:36 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:36 smithi097 ceph-mon[32602]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:36 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:36 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:36 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:38 smithi096 ceph-mon[29131]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:38.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:38.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:38.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:38 smithi097 ceph-mon[32602]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:38 smithi171 ceph-mon[33189]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:38.992 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.994+0000 7f4d617fa700 1 -- 172.21.15.171:0/1099260076 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d64063ad0 con 0x7f4d4c0afc00 2024-04-25T17:04:38.993 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2024-04-25T17:04:38.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.996+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d4c0afc00 msgr2=0x7f4d4c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:38.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.996+0000 7f4d6a384700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d4c0afc00 0x7f4d4c0b2110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d580038a0 tx=0x7f4d5801a040 comp rx=0 tx=0).stop 2024-04-25T17:04:38.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.996+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64104d00 msgr2=0x7f4d64104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:38.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.996+0000 7f4d6a384700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64104d00 0x7f4d64104560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400d980 tx=0x7f4d5400dc90 comp rx=0 tx=0).stop 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 shutdown_connections 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d64109fc0 0x7f4d64103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d64104d00 0x7f4d64104560 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d4c0afc00 0x7f4d4c0b2110 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 --2- 172.21.15.171:0/1099260076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d64107660 0x7f4d64102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 >> 172.21.15.171:0/1099260076 conn(0x7f4d640fe740 msgr2=0x7f4d641057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 shutdown_connections 2024-04-25T17:04:38.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:04:38.997+0000 7f4d6a384700 1 -- 172.21.15.171:0/1099260076 wait complete. 2024-04-25T17:04:39.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:39 smithi171 ceph-mon[33189]: from='osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T17:04:39.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:39 smithi171 ceph-mon[33189]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T17:04:39.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:39.278 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:39.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:39 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:39.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:39 smithi096 ceph-mon[29131]: from='osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T17:04:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:39 smithi096 ceph-mon[29131]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T17:04:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:39 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:39.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:39 smithi097 ceph-mon[32602]: from='osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T17:04:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:39 smithi097 ceph-mon[32602]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T17:04:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:39 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:04:39.582 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T17:04:39.584 INFO:tasks.cephadm:Deploying osd.6 on smithi096 with /dev/vg_nvme/lv_2... 2024-04-25T17:04:39.584 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T17:04:39.829 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: from='osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T17:04:40.193 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:40.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:40.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T17:04:40.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: from='osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:04:40.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T17:04:40.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:40.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:04:40.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:04:40.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T17:04:40.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: from='osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T17:04:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:40.694 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:04:40.525+0000 7f659a02f700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T17:04:40.694 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:04:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:04:40.531+0000 7f6591ddc700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:04:41.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:41 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:41.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:41 smithi171 ceph-mon[33189]: from='osd.5 ' entity='osd.5' 2024-04-25T17:04:41.337 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:41.337 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:41 smithi096 ceph-mon[29131]: from='osd.5 ' entity='osd.5' 2024-04-25T17:04:41.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:41 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:41.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:41 smithi097 ceph-mon[32602]: from='osd.5 ' entity='osd.5' 2024-04-25T17:04:42.065 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T17:04:42.066 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T17:04:42.066 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T17:04:42.066 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T17:04:42.066 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113656 s, 923 MB/s 2024-04-25T17:04:42.066 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T17:04:42.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:42 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:04:42.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:42 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:04:42.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:42 smithi096 ceph-mon[29131]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:42.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:42 smithi096 ceph-mon[29131]: osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] boot 2024-04-25T17:04:42.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:42 smithi096 ceph-mon[29131]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T17:04:42.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:42 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:42.443 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:42 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:04:42.444 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:42 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:04:42.444 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:42 smithi171 ceph-mon[33189]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:42.444 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:42 smithi171 ceph-mon[33189]: osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] boot 2024-04-25T17:04:42.444 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:42 smithi171 ceph-mon[33189]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T17:04:42.444 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:42 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:42 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:04:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:42 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:04:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:42 smithi097 ceph-mon[32602]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T17:04:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:42 smithi097 ceph-mon[32602]: osd.5 [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] boot 2024-04-25T17:04:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:42 smithi097 ceph-mon[32602]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T17:04:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:42 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:04:42.793 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-04-25T17:04:43.033 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:04:43.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T17:04:43.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi171 2024-04-25T17:04:43.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:43.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:43.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-25T17:04:43.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:43.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:43.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.412 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:43 smithi096 ceph-mon[29131]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:43.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi171 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:43 smithi097 ceph-mon[32602]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:43.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T17:04:43.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi171 2024-04-25T17:04:43.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:43.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:04:43.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-25T17:04:43.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:43.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:04:43.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:04:43.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:43 smithi171 ceph-mon[33189]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:44.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:44 smithi096 ceph-mon[29131]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T17:04:44.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:44 smithi171 ceph-mon[33189]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T17:04:44.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:44 smithi097 ceph-mon[32602]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T17:04:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 -- 172.21.15.96:0/855554533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220105ce0 msgr2=0x7fd22010c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 --2- 172.21.15.96:0/855554533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220105ce0 0x7fd22010c590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd21800b0a0 tx=0x7fd21800b3b0 comp rx=0 tx=0).stop 2024-04-25T17:04:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 -- 172.21.15.96:0/855554533 shutdown_connections 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 --2- 172.21.15.96:0/855554533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220105ce0 0x7fd22010c590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 --2- 172.21.15.96:0/855554533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd220104210 0x7fd220104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 --2- 172.21.15.96:0/855554533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd220105340 0x7fd220105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 -- 172.21.15.96:0/855554533 >> 172.21.15.96:0/855554533 conn(0x7fd2200ff870 msgr2=0x7fd220101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 -- 172.21.15.96:0/855554533 shutdown_connections 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 -- 172.21.15.96:0/855554533 wait complete. 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.734+0000 7fd226312700 1 Processor -- start 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd226312700 1 -- start start 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd226312700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220104210 0x7fd22019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd226312700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd220105340 0x7fd22019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd226312700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd220105ce0 0x7fd2201a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd226312700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd22010fc70 con 0x7fd220104210 2024-04-25T17:04:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd226312700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd22010faf0 con 0x7fd220105ce0 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd226312700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd22010fdf0 con 0x7fd220105340 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220104210 0x7fd22019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220104210 0x7fd22019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48296/0 (socket says 172.21.15.96:48296) 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 -- 172.21.15.96:0/14673687 learned_addr learned my addr 172.21.15.96:0/14673687 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21f7fe700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd220105340 0x7fd22019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd2248af700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd220105ce0 0x7fd2201a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 -- 172.21.15.96:0/14673687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd220105340 msgr2=0x7fd22019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd220105340 0x7fd22019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 -- 172.21.15.96:0/14673687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd220105ce0 msgr2=0x7fd2201a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd220105ce0 0x7fd2201a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.735+0000 7fd21ffff700 1 -- 172.21.15.96:0/14673687 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2201a2260 con 0x7fd220104210 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.736+0000 7fd21ffff700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220104210 0x7fd22019a700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c970 tx=0x7fd21000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.736+0000 7fd21d7fa700 1 -- 172.21.15.96:0/14673687 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210004220 con 0x7fd220104210 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.736+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2201a2550 con 0x7fd220104210 2024-04-25T17:04:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.736+0000 7fd21d7fa700 1 -- 172.21.15.96:0/14673687 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd210007db0 con 0x7fd220104210 2024-04-25T17:04:44.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.736+0000 7fd21d7fa700 1 -- 172.21.15.96:0/14673687 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210003d00 con 0x7fd220104210 2024-04-25T17:04:44.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.736+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2201a2ae0 con 0x7fd220104210 2024-04-25T17:04:44.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.737+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd220051190 con 0x7fd220104210 2024-04-25T17:04:44.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.738+0000 7fd21d7fa700 1 -- 172.21.15.96:0/14673687 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd210007900 con 0x7fd220104210 2024-04-25T17:04:44.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.739+0000 7fd21d7fa700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd2080afc00 0x7fd2080b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:04:44.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.739+0000 7fd21f7fe700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd2080afc00 0x7fd2080b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:04:44.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.739+0000 7fd21d7fa700 1 -- 172.21.15.96:0/14673687 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fd2100dddc0 con 0x7fd220104210 2024-04-25T17:04:44.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.739+0000 7fd21f7fe700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd2080afc00 0x7fd2080b2110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd214005950 tx=0x7fd214005ed0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:04:44.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.743+0000 7fd21d7fa700 1 -- 172.21.15.96:0/14673687 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd210014030 con 0x7fd220104210 2024-04-25T17:04:44.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:04:44.951+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd220063ad0 con 0x7fd2080afc00 2024-04-25T17:04:45.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:45 smithi096 ceph-mon[29131]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:04:45.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:04:45.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:04:45.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:45.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:45 smithi096 ceph-mon[29131]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T17:04:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:45 smithi097 ceph-mon[32602]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:04:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:04:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:04:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:45.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:45 smithi097 ceph-mon[32602]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T17:04:45.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:45 smithi171 ceph-mon[33189]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:04:45.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:04:45.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:04:45.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:45.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:45 smithi171 ceph-mon[33189]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-25T17:04:48.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:48 smithi096 ceph-mon[29131]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T17:04:48.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:48 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1886592691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f6a0ae2-7293-440b-9fcc-385526e69434"}]: dispatch 2024-04-25T17:04:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:48 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1886592691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f6a0ae2-7293-440b-9fcc-385526e69434"}]': finished 2024-04-25T17:04:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:48 smithi096 ceph-mon[29131]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T17:04:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:04:48.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:48 smithi097 ceph-mon[32602]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T17:04:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:48 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1886592691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f6a0ae2-7293-440b-9fcc-385526e69434"}]: dispatch 2024-04-25T17:04:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:48 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1886592691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f6a0ae2-7293-440b-9fcc-385526e69434"}]': finished 2024-04-25T17:04:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:48 smithi097 ceph-mon[32602]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T17:04:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:04:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:48 smithi171 ceph-mon[33189]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T17:04:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:48 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1886592691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f6a0ae2-7293-440b-9fcc-385526e69434"}]: dispatch 2024-04-25T17:04:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:48 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1886592691' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f6a0ae2-7293-440b-9fcc-385526e69434"}]': finished 2024-04-25T17:04:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:48 smithi171 ceph-mon[33189]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T17:04:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:04:49.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:49 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/102535369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:04:49.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:49 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/102535369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:04:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:49 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/102535369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:04:50.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:50 smithi096 ceph-mon[29131]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T17:04:50.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:50 smithi097 ceph-mon[32602]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T17:04:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:50 smithi171 ceph-mon[33189]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T17:04:52.190 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:52 smithi096 ceph-mon[29131]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:04:52.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:52 smithi097 ceph-mon[32602]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:04:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:52 smithi171 ceph-mon[33189]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:04:54.084 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:54 smithi096 ceph-mon[29131]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:04:54.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:54 smithi097 ceph-mon[32602]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:04:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:54 smithi171 ceph-mon[33189]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:04:56.118 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:56 smithi096 ceph-mon[29131]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:56.118 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T17:04:56.118 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:56.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:56 smithi097 ceph-mon[32602]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T17:04:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:56.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:56 smithi171 ceph-mon[33189]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:56.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T17:04:56.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:04:57.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:57 smithi096 ceph-mon[29131]: Deploying daemon osd.6 on smithi096 2024-04-25T17:04:57.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:57 smithi097 ceph-mon[32602]: Deploying daemon osd.6 on smithi096 2024-04-25T17:04:57.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:57 smithi171 ceph-mon[33189]: Deploying daemon osd.6 on smithi096 2024-04-25T17:04:58.300 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:04:58 smithi096 ceph-mon[29131]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:58.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:04:58 smithi097 ceph-mon[32602]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:04:58.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:04:58 smithi171 ceph-mon[33189]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:00.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:00 smithi096 ceph-mon[29131]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:00.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:00 smithi097 ceph-mon[32602]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:00 smithi171 ceph-mon[33189]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:02.270 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:02 smithi096 ceph-mon[29131]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:02.270 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:02 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:02.270 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:02 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:02.270 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:02 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:02 smithi097 ceph-mon[32602]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:02 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:02 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:02 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:02.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:02 smithi171 ceph-mon[33189]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:02.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:02 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:02.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:02 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:02.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:02 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:04.176 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:04 smithi096 ceph-mon[29131]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:04.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:04 smithi097 ceph-mon[32602]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:04.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:04 smithi171 ceph-mon[33189]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:05.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:05 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:05 smithi096 ceph-mon[29131]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:05 smithi096 ceph-mon[29131]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T17:05:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:05 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:05 smithi171 ceph-mon[33189]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:05.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:05 smithi171 ceph-mon[33189]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T17:05:05.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.607+0000 7fd21d7fa700 1 -- 172.21.15.96:0/14673687 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd220063ad0 con 0x7fd2080afc00 2024-04-25T17:05:05.606 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 6 on host 'smithi096' 2024-04-25T17:05:05.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd2080afc00 msgr2=0x7fd2080b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:05.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd2080afc00 0x7fd2080b2110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd214005950 tx=0x7fd214005ed0 comp rx=0 tx=0).stop 2024-04-25T17:05:05.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220104210 msgr2=0x7fd22019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220104210 0x7fd22019a700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c970 tx=0x7fd21000cd30 comp rx=0 tx=0).stop 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 shutdown_connections 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd220105ce0 0x7fd2201a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd2080afc00 0x7fd2080b2110 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd220104210 0x7fd22019a700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 --2- 172.21.15.96:0/14673687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd220105340 0x7fd22019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 >> 172.21.15.96:0/14673687 conn(0x7fd2200ff870 msgr2=0x7fd220108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 shutdown_connections 2024-04-25T17:05:05.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:05:05.609+0000 7fd226312700 1 -- 172.21.15.96:0/14673687 wait complete. 2024-04-25T17:05:05.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:05.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:05.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:05 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:05.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:05 smithi097 ceph-mon[32602]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:05.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:05 smithi097 ceph-mon[32602]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T17:05:06.407 DEBUG:teuthology.orchestra.run.smithi096:osd.6> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T17:05:06.410 INFO:tasks.cephadm:Deploying osd.7 on smithi097 with /dev/vg_nvme/lv_2... 2024-04-25T17:05:06.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T17:05:06.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:06 smithi171 ceph-mon[33189]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T17:05:06.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:06 smithi171 ceph-mon[33189]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T17:05:06.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:06 smithi171 ceph-mon[33189]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:05:06.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:06.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:06.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:06.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:06.638 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:06 smithi096 ceph-mon[29131]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T17:05:06.638 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:06 smithi096 ceph-mon[29131]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T17:05:06.639 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:06 smithi096 ceph-mon[29131]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:05:06.639 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:06.639 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:06.639 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:06.639 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:06.656 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:05:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:06 smithi097 ceph-mon[32602]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T17:05:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:06 smithi097 ceph-mon[32602]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T17:05:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:06 smithi097 ceph-mon[32602]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:05:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:07 smithi097 ceph-mon[32602]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:05:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:07 smithi097 ceph-mon[32602]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T17:05:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:07 smithi097 ceph-mon[32602]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:07.603 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:07 smithi096 ceph-mon[29131]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:05:07.603 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:07 smithi096 ceph-mon[29131]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T17:05:07.603 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:07.603 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:07.603 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:07 smithi096 ceph-mon[29131]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:07.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:07 smithi171 ceph-mon[33189]: from='osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:05:07.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:07 smithi171 ceph-mon[33189]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T17:05:07.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:07.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:07.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:07 smithi171 ceph-mon[33189]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T17:05:08.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:08 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:05:08.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:08 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:05:08.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:08 smithi096 ceph-mon[29131]: osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] boot 2024-04-25T17:05:08.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:08 smithi096 ceph-mon[29131]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T17:05:08.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:08 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:05:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:08 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:05:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:08 smithi097 ceph-mon[32602]: osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] boot 2024-04-25T17:05:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:08 smithi097 ceph-mon[32602]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T17:05:08.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:08.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:08 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:05:08.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:08 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:05:08.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:08 smithi171 ceph-mon[33189]: osd.6 [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] boot 2024-04-25T17:05:08.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:08 smithi171 ceph-mon[33189]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T17:05:08.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:05:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T17:05:08.841 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T17:05:08.842 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T17:05:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T17:05:08.842 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013046 s, 804 MB/s 2024-04-25T17:05:08.842 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T17:05:09.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:09 smithi097 ceph-mon[32602]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T17:05:09.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:09 smithi097 ceph-mon[32602]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:09.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-04-25T17:05:09.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:09 smithi096 ceph-mon[29131]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T17:05:09.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:09 smithi096 ceph-mon[29131]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:09.761 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:05:09.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:09 smithi171 ceph-mon[33189]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T17:05:09.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:09 smithi171 ceph-mon[33189]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi096 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T17:05:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:10.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:10.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi096 2024-04-25T17:05:10.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi096 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:10.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:10.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:11.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 -- 172.21.15.97:0/1873431960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b41057d0 msgr2=0x7f53b4105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/1873431960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b41057d0 0x7f53b4105be0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009990 tx=0x7f53a4009ca0 comp rx=0 tx=0).stop 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 -- 172.21.15.97:0/1873431960 shutdown_connections 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/1873431960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53b4102b70 0x7f53b410c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/1873431960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b41057d0 0x7f53b4105be0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/1873431960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b4102220 0x7f53b4102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 -- 172.21.15.97:0/1873431960 >> 172.21.15.97:0/1873431960 conn(0x7f53b40fdef0 msgr2=0x7f53b41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 -- 172.21.15.97:0/1873431960 shutdown_connections 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.406+0000 7f53ba3b8700 1 -- 172.21.15.97:0/1873431960 wait complete. 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 Processor -- start 2024-04-25T17:05:11.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 -- start start 2024-04-25T17:05:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b4102220 0x7f53b4104da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b4102b70 0x7f53b4103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53b41057d0 0x7f53b4103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:11.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53b410fc80 con 0x7f53b4102b70 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53b410fb00 con 0x7f53b4102220 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53ba3b8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f53b410fe00 con 0x7f53b41057d0 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b4102220 0x7f53b4104da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b4102220 0x7f53b4104da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37710/0 (socket says 172.21.15.97:37710) 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 -- 172.21.15.97:0/2555976663 learned_addr learned my addr 172.21.15.97:0/2555976663 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b37fe700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b4102b70 0x7f53b4103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b8955700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53b41057d0 0x7f53b4103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 -- 172.21.15.97:0/2555976663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53b41057d0 msgr2=0x7f53b4103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53b41057d0 0x7f53b4103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 -- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b4102b70 msgr2=0x7f53b4103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b4102b70 0x7f53b4103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b3fff700 1 -- 172.21.15.97:0/2555976663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b4104060 con 0x7f53b4102220 2024-04-25T17:05:11.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.407+0000 7f53b8955700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53b41057d0 0x7f53b4103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:05:11.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.408+0000 7f53b3fff700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b4102220 0x7f53b4104da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000c00 tx=0x7f53a40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:05:11.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.408+0000 7f53b17fa700 1 -- 172.21.15.97:0/2555976663 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4003c90 con 0x7f53b4102220 2024-04-25T17:05:11.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.408+0000 7f53b17fa700 1 -- 172.21.15.97:0/2555976663 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a4004830 con 0x7f53b4102220 2024-04-25T17:05:11.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.408+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b41a48c0 con 0x7f53b4102220 2024-04-25T17:05:11.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.408+0000 7f53b17fa700 1 -- 172.21.15.97:0/2555976663 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4005470 con 0x7f53b4102220 2024-04-25T17:05:11.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.408+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b41a4c60 con 0x7f53b4102220 2024-04-25T17:05:11.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.409+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b4051190 con 0x7f53b4102220 2024-04-25T17:05:11.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.410+0000 7f53b17fa700 1 -- 172.21.15.97:0/2555976663 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f53a40049e0 con 0x7f53b4102220 2024-04-25T17:05:11.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.411+0000 7f53b17fa700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f539c0afb80 0x7f539c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:11.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.411+0000 7f53b17fa700 1 -- 172.21.15.97:0/2555976663 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f53a40def00 con 0x7f53b4102220 2024-04-25T17:05:11.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.411+0000 7f53b37fe700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f539c0afb80 0x7f539c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:11.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.412+0000 7f53b37fe700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f539c0afb80 0x7f539c0b2090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53b4104750 tx=0x7f53a800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:05:11.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.416+0000 7f53b17fa700 1 -- 172.21.15.97:0/2555976663 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53a4013070 con 0x7f53b4102220 2024-04-25T17:05:11.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:11.624+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f53b4063ad0 con 0x7f539c0afb80 2024-04-25T17:05:11.732 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:11 smithi097 ceph-mon[32602]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:11.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:11 smithi171 ceph-mon[33189]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:11.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:11 smithi096 ceph-mon[29131]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:12.553 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:12 smithi097 ceph-mon[32602]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:05:12.553 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:12 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:05:12.553 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:12 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:05:12.553 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:12 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:12.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:12 smithi171 ceph-mon[33189]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:05:12.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:12 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:05:12.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:12 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:05:12.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:12 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:12 smithi096 ceph-mon[29131]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:05:12.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:12 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:05:12.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:12 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:05:12.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:12 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:13.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:13 smithi097 ceph-mon[32602]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:13.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:13 smithi171 ceph-mon[33189]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:13.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:13 smithi096 ceph-mon[29131]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:14.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:14 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/576932075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]: dispatch 2024-04-25T17:05:14.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:14 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]: dispatch 2024-04-25T17:05:14.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:14 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]': finished 2024-04-25T17:05:14.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:14 smithi097 ceph-mon[32602]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T17:05:14.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:14 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:14.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:14 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/576932075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]: dispatch 2024-04-25T17:05:14.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:14 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]: dispatch 2024-04-25T17:05:14.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:14 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]': finished 2024-04-25T17:05:14.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:14 smithi096 ceph-mon[29131]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T17:05:14.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:14 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:15.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:14 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/576932075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]: dispatch 2024-04-25T17:05:15.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:14 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]: dispatch 2024-04-25T17:05:15.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:14 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2281b753-0cf4-41de-b12c-9dda1afa5c71"}]': finished 2024-04-25T17:05:15.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:14 smithi171 ceph-mon[33189]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T17:05:15.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:14 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:15 smithi096 ceph-mon[29131]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:15 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/498746213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:05:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:15 smithi097 ceph-mon[32602]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:15 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/498746213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:05:16.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:15 smithi171 ceph-mon[33189]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:16.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:15 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/498746213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:05:18.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:18 smithi096 ceph-mon[29131]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:18.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:18 smithi097 ceph-mon[32602]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:18.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:18 smithi171 ceph-mon[33189]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:20.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:20 smithi097 ceph-mon[32602]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:20.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:20 smithi096 ceph-mon[29131]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:20.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:20 smithi171 ceph-mon[33189]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:22.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:22 smithi096 ceph-mon[29131]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:22.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:22 smithi097 ceph-mon[32602]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:22.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:22 smithi171 ceph-mon[33189]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:23 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T17:05:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:23 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:23.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:23 smithi096 ceph-mon[29131]: Deploying daemon osd.7 on smithi097 2024-04-25T17:05:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:23 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T17:05:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:23 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:23 smithi097 ceph-mon[32602]: Deploying daemon osd.7 on smithi097 2024-04-25T17:05:23.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:23 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T17:05:23.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:23 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:23.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:23 smithi171 ceph-mon[33189]: Deploying daemon osd.7 on smithi097 2024-04-25T17:05:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:24 smithi097 ceph-mon[32602]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:24.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:24 smithi096 ceph-mon[29131]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:24.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:24 smithi171 ceph-mon[33189]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:26.225 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:26 smithi097 ceph-mon[32602]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:26.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:26 smithi096 ceph-mon[29131]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:26.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:26 smithi171 ceph-mon[33189]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:28.408 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:28 smithi097 ceph-mon[32602]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:28.408 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:28 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:28.408 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:28 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:28.409 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:28 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:28.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:28 smithi096 ceph-mon[29131]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:28.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:28 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:28.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:28 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:28.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:28 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:28 smithi171 ceph-mon[33189]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:28 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:28 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:28 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:30.341 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:30 smithi097 ceph-mon[32602]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:30.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:30 smithi096 ceph-mon[29131]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:30.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:30 smithi171 ceph-mon[33189]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.095+0000 7f53b17fa700 1 -- 172.21.15.97:0/2555976663 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f53b4063ad0 con 0x7f539c0afb80 2024-04-25T17:05:32.095 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 7 on host 'smithi097' 2024-04-25T17:05:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f539c0afb80 msgr2=0x7f539c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f539c0afb80 0x7f539c0b2090 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53b4104750 tx=0x7f53a800a380 comp rx=0 tx=0).stop 2024-04-25T17:05:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b4102220 msgr2=0x7f53b4104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b4102220 0x7f53b4104da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000c00 tx=0x7f53a40047b0 comp rx=0 tx=0).stop 2024-04-25T17:05:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 shutdown_connections 2024-04-25T17:05:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53b41057d0 0x7f53b4103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:32.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53b4102220 0x7f53b4104da0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f539c0afb80 0x7f539c0b2090 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 --2- 172.21.15.97:0/2555976663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53b4102b70 0x7f53b4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 >> 172.21.15.97:0/2555976663 conn(0x7f53b40fdef0 msgr2=0x7f53b4108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:05:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 shutdown_connections 2024-04-25T17:05:32.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:05:32.098+0000 7f53ba3b8700 1 -- 172.21.15.97:0/2555976663 wait complete. 2024-04-25T17:05:32.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:32.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:32.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:32.423 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T17:05:32.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T17:05:32.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.424 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T17:05:32.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T17:05:32.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:32.791 DEBUG:teuthology.orchestra.run.smithi097:osd.7> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T17:05:32.793 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2024-04-25T17:05:32.794 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T17:05:33.040 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:05:33.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:33.301 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-mon[32602]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T17:05:33.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-mon[32602]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T17:05:33.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:33.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-mon[32602]: from='osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:05:33.302 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-mon[32602]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:05:33.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:33 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:33.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:33 smithi171 ceph-mon[33189]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T17:05:33.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:33 smithi171 ceph-mon[33189]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T17:05:33.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:33 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:33.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:33 smithi171 ceph-mon[33189]: from='osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:05:33.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:33 smithi171 ceph-mon[33189]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:05:33.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:33 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:33.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:33 smithi096 ceph-mon[29131]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T17:05:33.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:33 smithi096 ceph-mon[29131]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T17:05:33.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:33 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:33.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:33 smithi096 ceph-mon[29131]: from='osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:05:33.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:33 smithi096 ceph-mon[29131]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:05:33.983 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:05:33.560+0000 7f725225f700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T17:05:33.983 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:05:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:05:33.570+0000 7f724a00c700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:05:34.436 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:34.436 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:05:34.436 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T17:05:34.436 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:34 smithi171 ceph-mon[33189]: from='osd.7 ' entity='osd.7' 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T17:05:34.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:34 smithi096 ceph-mon[29131]: from='osd.7 ' entity='osd.7' 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: osdmap e47: 8 total, 7 up, 8 in 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:34 smithi097 ceph-mon[32602]: from='osd.7 ' entity='osd.7' 2024-04-25T17:05:34.993 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T17:05:34.993 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T17:05:34.993 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T17:05:34.993 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T17:05:34.993 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011371 s, 922 MB/s 2024-04-25T17:05:34.993 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T17:05:35.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:35 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:05:35.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:35 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:05:35.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:35 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:35.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:35 smithi097 ceph-mon[32602]: osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] boot 2024-04-25T17:05:35.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:35 smithi097 ceph-mon[32602]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T17:05:35.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:35 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:35.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:35 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:05:35.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:35 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:05:35.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:35 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:35.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:35 smithi171 ceph-mon[33189]: osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] boot 2024-04-25T17:05:35.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:35 smithi171 ceph-mon[33189]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T17:05:35.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:35 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:35.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:35 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:05:35.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:35 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:05:35.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:35 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:35.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:35 smithi096 ceph-mon[29131]: osd.7 [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] boot 2024-04-25T17:05:35.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:35 smithi096 ceph-mon[29131]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T17:05:35.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:35 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:05:35.627 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-04-25T17:05:35.880 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:05:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:36 smithi097 ceph-mon[32602]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:36.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:36 smithi097 ceph-mon[32602]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T17:05:36.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:36 smithi171 ceph-mon[33189]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:36.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:36 smithi171 ceph-mon[33189]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T17:05:36.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:36 smithi096 ceph-mon[29131]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-25T17:05:36.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:36 smithi096 ceph-mon[29131]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/818071902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 msgr2=0x7f2ec810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/818071902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec810b490 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000b0a0 tx=0x7f2ec000b3b0 comp rx=0 tx=0).stop 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/818071902 shutdown_connections 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/818071902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec810b490 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/818071902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8103110 0x7f2ec8103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/818071902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec8104240 0x7f2ec8104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/818071902 >> 172.21.15.171:0/818071902 conn(0x7f2ec80fe750 msgr2=0x7f2ec8100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.430+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/818071902 shutdown_connections 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/818071902 wait complete. 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 Processor -- start 2024-04-25T17:05:37.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 -- start start 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8103110 0x7f2ec8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec8104240 0x7f2ec8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810eb80 con 0x7f2ec8104240 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810ea00 con 0x7f2ec8103110 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ecfbd3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec810ed00 con 0x7f2ec8104be0 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.431+0000 7f2ece170700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:37.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec81a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:37734/0 (socket says 172.21.15.171:37734) 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 -- 172.21.15.171:0/3389165155 learned_addr learned my addr 172.21.15.171:0/3389165155 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ecd96f700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8103110 0x7f2ec8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ecd16e700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec8104240 0x7f2ec8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 -- 172.21.15.171:0/3389165155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8103110 msgr2=0x7f2ec8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8103110 0x7f2ec8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 -- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec8104240 msgr2=0x7f2ec8199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec8104240 0x7f2ec8199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 -- 172.21.15.171:0/3389165155 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec81a11c0 con 0x7f2ec8104be0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ece170700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec81a0b40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000b070 tx=0x7f2ec0004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ebeffd700 1 -- 172.21.15.171:0/3389165155 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0021070 con 0x7f2ec8104be0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec81a1450 con 0x7f2ec8104be0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ebeffd700 1 -- 172.21.15.171:0/3389165155 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec0007b80 con 0x7f2ec8104be0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ebeffd700 1 -- 172.21.15.171:0/3389165155 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00052a0 con 0x7f2ec8104be0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.432+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec81a1990 con 0x7f2ec8104be0 2024-04-25T17:05:37.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.433+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec8051190 con 0x7f2ec8104be0 2024-04-25T17:05:37.434 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.435+0000 7f2ebeffd700 1 -- 172.21.15.171:0/3389165155 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2ec0024070 con 0x7f2ec8104be0 2024-04-25T17:05:37.435 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.436+0000 7f2ebeffd700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2eb40afb80 0x7f2eb40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:05:37.435 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.436+0000 7f2ebeffd700 1 -- 172.21.15.171:0/3389165155 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f2ec000f020 con 0x7f2ec8104be0 2024-04-25T17:05:37.435 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.436+0000 7f2ecd96f700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2eb40afb80 0x7f2eb40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:05:37.435 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.437+0000 7f2ecd96f700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2eb40afb80 0x7f2eb40b2090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8104140 tx=0x7f2eb800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:05:37.438 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.440+0000 7f2ebeffd700 1 -- 172.21.15.171:0/3389165155 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ec002b020 con 0x7f2ec8104be0 2024-04-25T17:05:37.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T17:05:37.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi097 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:37 smithi171 ceph-mon[33189]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:37.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:37.644+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2ec8063ad0 con 0x7f2eb40afb80 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi097 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:37.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:37.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:37 smithi096 ceph-mon[29131]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: osdmap e50: 8 total, 8 up, 8 in 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi097 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:37.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:37.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:37.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:37 smithi097 ceph-mon[32602]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:38 smithi171 ceph-mon[33189]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:05:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:05:38.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:05:38.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:38 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:38.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:38 smithi096 ceph-mon[29131]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:05:38.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:05:38.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:05:38.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:38 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:38.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:38 smithi097 ceph-mon[32602]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:05:38.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:05:38.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:05:38.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:38 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:39.625 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:39 smithi171 ceph-mon[33189]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:39 smithi096 ceph-mon[29131]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:39.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:39 smithi097 ceph-mon[32602]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:40.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:40 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/3203194514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]: dispatch 2024-04-25T17:05:40.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:40 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]: dispatch 2024-04-25T17:05:40.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:40 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]': finished 2024-04-25T17:05:40.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:40 smithi171 ceph-mon[33189]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T17:05:40.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:40.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:40 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/3203194514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]: dispatch 2024-04-25T17:05:40.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:40 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]: dispatch 2024-04-25T17:05:40.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:40 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]': finished 2024-04-25T17:05:40.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:40 smithi096 ceph-mon[29131]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T17:05:40.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:40 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/3203194514' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]: dispatch 2024-04-25T17:05:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:40 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]: dispatch 2024-04-25T17:05:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:40 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "521eeba2-7a5c-44a5-839e-d57d391c2a75"}]': finished 2024-04-25T17:05:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:40 smithi097 ceph-mon[32602]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T17:05:40.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:41.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:41 smithi096 ceph-mon[29131]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T17:05:41.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:41 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/1568303620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:05:41.982 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:41 smithi097 ceph-mon[32602]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T17:05:41.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:41 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/1568303620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:05:42.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:41 smithi171 ceph-mon[33189]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T17:05:42.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:41 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/1568303620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:05:44.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:44 smithi096 ceph-mon[29131]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T17:05:44.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:44 smithi097 ceph-mon[32602]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T17:05:44.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:44 smithi171 ceph-mon[33189]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T17:05:46.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:46 smithi171 ceph-mon[33189]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T17:05:46.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:46 smithi096 ceph-mon[29131]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T17:05:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:46 smithi097 ceph-mon[32602]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T17:05:48.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:48 smithi171 ceph-mon[33189]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:05:48.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T17:05:48.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:48.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:48 smithi096 ceph-mon[29131]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:05:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T17:05:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:48 smithi097 ceph-mon[32602]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:05:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T17:05:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:49.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:49 smithi171 ceph-mon[33189]: Deploying daemon osd.8 on smithi171 2024-04-25T17:05:49.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:49 smithi096 ceph-mon[29131]: Deploying daemon osd.8 on smithi171 2024-04-25T17:05:49.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:49 smithi097 ceph-mon[32602]: Deploying daemon osd.8 on smithi171 2024-04-25T17:05:50.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:50 smithi096 ceph-mon[29131]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:05:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:50 smithi097 ceph-mon[32602]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:05:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:50 smithi171 ceph-mon[33189]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:05:52.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:52 smithi171 ceph-mon[33189]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:52.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:52 smithi096 ceph-mon[29131]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:52 smithi097 ceph-mon[32602]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:54.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:54 smithi171 ceph-mon[33189]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:54.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:54.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:54.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:54.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:54 smithi096 ceph-mon[29131]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:54 smithi097 ceph-mon[32602]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:56.401 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:56 smithi171 ceph-mon[33189]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:56.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:56 smithi096 ceph-mon[29131]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:56 smithi097 ceph-mon[32602]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:57.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.515+0000 7f2ebeffd700 1 -- 172.21.15.171:0/3389165155 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ec8063ad0 con 0x7f2eb40afb80 2024-04-25T17:05:57.515 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2eb40afb80 msgr2=0x7f2eb40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2eb40afb80 0x7f2eb40b2090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8104140 tx=0x7f2eb800b410 comp rx=0 tx=0).stop 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 msgr2=0x7f2ec81a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec81a0b40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000b070 tx=0x7f2ec0004ce0 comp rx=0 tx=0).stop 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 shutdown_connections 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ec8104be0 0x7f2ec81a0b40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ec8103110 0x7f2ec8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2eb40afb80 0x7f2eb40b2090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 --2- 172.21.15.171:0/3389165155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ec8104240 0x7f2ec8199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 >> 172.21.15.171:0/3389165155 conn(0x7f2ec80fe750 msgr2=0x7f2ec8107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 shutdown_connections 2024-04-25T17:05:57.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:05:57.518+0000 7f2ecfbd3700 1 -- 172.21.15.171:0/3389165155 wait complete. 2024-04-25T17:05:57.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:57.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:57.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T17:05:57.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T17:05:57.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:57.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:05:57.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:05:58.161 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T17:05:58.163 INFO:tasks.cephadm:Deploying osd.9 on smithi096 with /dev/vg_nvme/lv_1... 2024-04-25T17:05:58.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T17:05:58.418 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:05:58.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:58 smithi171 ceph-mon[33189]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T17:05:58.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:58 smithi171 ceph-mon[33189]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T17:05:58.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:58 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:58.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:58 smithi171 ceph-mon[33189]: from='osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:05:58.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:58 smithi171 ceph-mon[33189]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:05:58.801 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:58 smithi096 ceph-mon[29131]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T17:05:58.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:58 smithi096 ceph-mon[29131]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T17:05:58.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:58 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:58.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:58 smithi096 ceph-mon[29131]: from='osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:05:58.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:58 smithi096 ceph-mon[29131]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:05:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:58 smithi097 ceph-mon[32602]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T17:05:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:58 smithi097 ceph-mon[32602]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T17:05:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:58 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:58 smithi097 ceph-mon[32602]: from='osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:05:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:58 smithi097 ceph-mon[32602]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:05:59.362 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:05:59.103+0000 7f0fecf09700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T17:05:59.362 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:05:59.113+0000 7f0fe4cb6700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:05:59.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:59 smithi096 ceph-mon[29131]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:05:59.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:59 smithi096 ceph-mon[29131]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T17:05:59.848 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:59 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:59.848 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:59 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:59.848 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:59 smithi096 ceph-mon[29131]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:59.848 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:05:59 smithi096 ceph-mon[29131]: from='osd.8 ' entity='osd.8' 2024-04-25T17:05:59.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:59 smithi097 ceph-mon[32602]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:05:59.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:59 smithi097 ceph-mon[32602]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T17:05:59.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:59 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:59.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:59 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:05:59.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:59 smithi097 ceph-mon[32602]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:05:59.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:05:59 smithi097 ceph-mon[32602]: from='osd.8 ' entity='osd.8' 2024-04-25T17:06:00.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-mon[33189]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:06:00.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-mon[33189]: osdmap e53: 9 total, 8 up, 9 in 2024-04-25T17:06:00.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:00.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:00.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-mon[33189]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T17:06:00.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:05:59 smithi171 ceph-mon[33189]: from='osd.8 ' entity='osd.8' 2024-04-25T17:06:00.562 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T17:06:00.562 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T17:06:00.563 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T17:06:00.563 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T17:06:00.563 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132623 s, 791 MB/s 2024-04-25T17:06:00.563 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T17:06:00.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:00 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:06:00.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:00 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:06:00.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:00.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:00 smithi171 ceph-mon[33189]: osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] boot 2024-04-25T17:06:00.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:00 smithi171 ceph-mon[33189]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T17:06:00.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:00.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:00 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:06:00.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:00 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:06:00.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:00.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:00 smithi096 ceph-mon[29131]: osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] boot 2024-04-25T17:06:00.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:00 smithi096 ceph-mon[29131]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T17:06:00.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:00 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:06:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:00 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:06:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:00 smithi097 ceph-mon[32602]: osd.8 [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] boot 2024-04-25T17:06:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:00 smithi097 ceph-mon[32602]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T17:06:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:06:01.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-04-25T17:06:01.563 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:01.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:01.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:01.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:01.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T17:06:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:01.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:01.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:01.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:01.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:01.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:01.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:01.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:02.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T17:06:02.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:02.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:02.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:02.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:02.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:02.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:02.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:02.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:02.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:02.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:02 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi171 2024-04-25T17:06:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:02 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-25T17:06:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:02 smithi097 ceph-mon[32602]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T17:06:03.021 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:02 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi171 2024-04-25T17:06:03.021 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:02 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-25T17:06:03.022 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:02 smithi096 ceph-mon[29131]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T17:06:03.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:02 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi171 2024-04-25T17:06:03.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:02 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-25T17:06:03.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:02 smithi171 ceph-mon[33189]: osdmap e56: 9 total, 9 up, 9 in 2024-04-25T17:06:03.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 -- 172.21.15.96:0/548359590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40104bd0 msgr2=0x7fbe4010b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:03.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 --2- 172.21.15.96:0/548359590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40104bd0 0x7fbe4010b480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800b0a0 tx=0x7fbe3800b3b0 comp rx=0 tx=0).stop 2024-04-25T17:06:03.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 -- 172.21.15.96:0/548359590 shutdown_connections 2024-04-25T17:06:03.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 --2- 172.21.15.96:0/548359590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40104bd0 0x7fbe4010b480 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:03.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 --2- 172.21.15.96:0/548359590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40103100 0x7fbe40103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:03.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 --2- 172.21.15.96:0/548359590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe40104230 0x7fbe40104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:03.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 -- 172.21.15.96:0/548359590 >> 172.21.15.96:0/548359590 conn(0x7fbe400fe760 msgr2=0x7fbe40100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:06:03.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 -- 172.21.15.96:0/548359590 shutdown_connections 2024-04-25T17:06:03.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.258+0000 7fbe4704d700 1 -- 172.21.15.96:0/548359590 wait complete. 2024-04-25T17:06:03.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 Processor -- start 2024-04-25T17:06:03.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 -- start start 2024-04-25T17:06:03.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40103100 0x7fbe40199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:03.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe40104230 0x7fbe40199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:03.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40104bd0 0x7fbe401a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:03.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4010ebf0 con 0x7fbe40103100 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4010ea70 con 0x7fbe40104bd0 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe4704d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4010ed70 con 0x7fbe40104230 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40103100 0x7fbe40199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40103100 0x7fbe40199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58542/0 (socket says 172.21.15.96:58542) 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 -- 172.21.15.96:0/267647547 learned_addr learned my addr 172.21.15.96:0/267647547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe3ffff700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe40104230 0x7fbe40199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe455ea700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40104bd0 0x7fbe401a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 -- 172.21.15.96:0/267647547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe40104230 msgr2=0x7fbe40199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe40104230 0x7fbe40199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 -- 172.21.15.96:0/267647547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40104bd0 msgr2=0x7fbe401a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40104bd0 0x7fbe401a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe44de9700 1 -- 172.21.15.96:0/267647547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe401a1150 con 0x7fbe40103100 2024-04-25T17:06:03.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe3ffff700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe40104230 0x7fbe40199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.259+0000 7fbe455ea700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40104bd0 0x7fbe401a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.260+0000 7fbe44de9700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40103100 0x7fbe40199680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c970 tx=0x7fbe3400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.260+0000 7fbe3dffb700 1 -- 172.21.15.96:0/267647547 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34007b00 con 0x7fbe40103100 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.260+0000 7fbe3dffb700 1 -- 172.21.15.96:0/267647547 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe34007ca0 con 0x7fbe40103100 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.260+0000 7fbe3dffb700 1 -- 172.21.15.96:0/267647547 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34018660 con 0x7fbe40103100 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.260+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe401a1440 con 0x7fbe40103100 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.260+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401a1a50 con 0x7fbe40103100 2024-04-25T17:06:03.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.261+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe40051190 con 0x7fbe40103100 2024-04-25T17:06:03.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.262+0000 7fbe3dffb700 1 -- 172.21.15.96:0/267647547 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbe3400f450 con 0x7fbe40103100 2024-04-25T17:06:03.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.263+0000 7fbe3dffb700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbe300afc00 0x7fbe300b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:03.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.263+0000 7fbe3ffff700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbe300afc00 0x7fbe300b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:03.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.263+0000 7fbe3dffb700 1 -- 172.21.15.96:0/267647547 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7fbe340de280 con 0x7fbe40103100 2024-04-25T17:06:03.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.263+0000 7fbe3ffff700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbe300afc00 0x7fbe300b2110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe4019a730 tx=0x7fbe2c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:06:03.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.267+0000 7fbe3dffb700 1 -- 172.21.15.96:0/267647547 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe34014030 con 0x7fbe40103100 2024-04-25T17:06:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:03.476+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbe40063ad0 con 0x7fbe300afc00 2024-04-25T17:06:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:03 smithi096 ceph-mon[29131]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:03.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:03 smithi097 ceph-mon[32602]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:03 smithi171 ceph-mon[33189]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:04 smithi097 ceph-mon[32602]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:04 smithi171 ceph-mon[33189]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:05.113 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:04 smithi096 ceph-mon[29131]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:05.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:05 smithi097 ceph-mon[32602]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:06.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:05 smithi171 ceph-mon[33189]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:06.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:05 smithi096 ceph-mon[29131]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:06 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3729099030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d96f5d9-4be8-445e-a878-b9c296673d1f"}]: dispatch 2024-04-25T17:06:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:06 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3729099030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d96f5d9-4be8-445e-a878-b9c296673d1f"}]': finished 2024-04-25T17:06:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:06 smithi097 ceph-mon[32602]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T17:06:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:07.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:06 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3729099030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d96f5d9-4be8-445e-a878-b9c296673d1f"}]: dispatch 2024-04-25T17:06:07.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:06 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3729099030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d96f5d9-4be8-445e-a878-b9c296673d1f"}]': finished 2024-04-25T17:06:07.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:06 smithi171 ceph-mon[33189]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T17:06:07.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:07.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:06 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3729099030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d96f5d9-4be8-445e-a878-b9c296673d1f"}]: dispatch 2024-04-25T17:06:07.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:06 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3729099030' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d96f5d9-4be8-445e-a878-b9c296673d1f"}]': finished 2024-04-25T17:06:07.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:06 smithi096 ceph-mon[29131]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T17:06:07.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:07.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:07 smithi097 ceph-mon[32602]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T17:06:07.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:07 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1307387852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:06:08.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:07 smithi171 ceph-mon[33189]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T17:06:08.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:07 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1307387852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:06:08.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:07 smithi096 ceph-mon[29131]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T17:06:08.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:07 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1307387852' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:06:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:10 smithi096 ceph-mon[29131]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:06:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:10 smithi097 ceph-mon[32602]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:06:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:10 smithi171 ceph-mon[33189]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:06:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:12 smithi096 ceph-mon[29131]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T17:06:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:12 smithi097 ceph-mon[32602]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T17:06:12.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:12 smithi171 ceph-mon[33189]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T17:06:14.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:14 smithi096 ceph-mon[29131]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:06:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:14 smithi097 ceph-mon[32602]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:06:14.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:14 smithi171 ceph-mon[33189]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:06:15.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T17:06:15.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:15.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T17:06:15.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:15.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T17:06:15.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:16 smithi096 ceph-mon[29131]: Deploying daemon osd.9 on smithi096 2024-04-25T17:06:16.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:16 smithi096 ceph-mon[29131]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:06:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:16 smithi097 ceph-mon[32602]: Deploying daemon osd.9 on smithi096 2024-04-25T17:06:16.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:16 smithi097 ceph-mon[32602]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:06:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:16 smithi171 ceph-mon[33189]: Deploying daemon osd.9 on smithi096 2024-04-25T17:06:16.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:16 smithi171 ceph-mon[33189]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:06:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:18 smithi096 ceph-mon[29131]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:18 smithi097 ceph-mon[32602]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:18.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:18 smithi171 ceph-mon[33189]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:20.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:20 smithi096 ceph-mon[29131]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:20 smithi097 ceph-mon[32602]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:20.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:20 smithi171 ceph-mon[33189]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:21.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:21 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:21.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:21 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:21.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:21 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:21.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:21 smithi096 ceph-mon[29131]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:21.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:21 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:21.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:21 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:21.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:21 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:21.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:21 smithi097 ceph-mon[32602]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:22.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:21 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:22.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:21 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:22.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:21 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:22.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:21 smithi171 ceph-mon[33189]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:24 smithi096 ceph-mon[29131]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:24 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:24 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:24 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:24 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:24 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:24 smithi097 ceph-mon[32602]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:24 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:24 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:24 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:24 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:24.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:24 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:24 smithi171 ceph-mon[33189]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:24.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:24 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:24 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:24 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:24 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:24 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.847+0000 7fbe3dffb700 1 -- 172.21.15.96:0/267647547 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbe40063ad0 con 0x7fbe300afc00 2024-04-25T17:06:24.846 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 9 on host 'smithi096' 2024-04-25T17:06:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbe300afc00 msgr2=0x7fbe300b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbe300afc00 0x7fbe300b2110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe4019a730 tx=0x7fbe2c008040 comp rx=0 tx=0).stop 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40103100 msgr2=0x7fbe40199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40103100 0x7fbe40199680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c970 tx=0x7fbe3400cd30 comp rx=0 tx=0).stop 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 shutdown_connections 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe40104bd0 0x7fbe401a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbe300afc00 0x7fbe300b2110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe40103100 0x7fbe40199680 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 --2- 172.21.15.96:0/267647547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe40104230 0x7fbe40199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 >> 172.21.15.96:0/267647547 conn(0x7fbe400fe760 msgr2=0x7fbe40107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 shutdown_connections 2024-04-25T17:06:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:06:24.850+0000 7fbe4704d700 1 -- 172.21.15.96:0/267647547 wait complete. 2024-04-25T17:06:25.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:25 smithi096 ceph-mon[29131]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T17:06:25.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:25.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:25.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:25 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:25 smithi097 ceph-mon[32602]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T17:06:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:25 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:25 smithi171 ceph-mon[33189]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T17:06:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:25.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:25 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:25.641 DEBUG:teuthology.orchestra.run.smithi096:osd.9> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T17:06:25.647 INFO:tasks.cephadm:Deploying osd.10 on smithi097 with /dev/vg_nvme/lv_1... 2024-04-25T17:06:25.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T17:06:25.895 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:06:26.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:26 smithi096 ceph-mon[29131]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:26.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:26 smithi096 ceph-mon[29131]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T17:06:26.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:26 smithi096 ceph-mon[29131]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T17:06:26.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:26 smithi096 ceph-mon[29131]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:06:26.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:26 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:26.333 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:26 smithi097 ceph-mon[32602]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:26.333 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:26 smithi097 ceph-mon[32602]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T17:06:26.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:26 smithi097 ceph-mon[32602]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T17:06:26.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:26 smithi097 ceph-mon[32602]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:06:26.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:26 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:26.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:26 smithi171 ceph-mon[33189]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:26 smithi171 ceph-mon[33189]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T17:06:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:26 smithi171 ceph-mon[33189]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T17:06:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:26 smithi171 ceph-mon[33189]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T17:06:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:26 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:26.938 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:06:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:06:26.567+0000 7fc7bcf6d700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T17:06:26.938 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:06:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:06:26.578+0000 7fc7b4d1a700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:06:27.288 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:27 smithi096 ceph-mon[29131]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:06:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:27 smithi096 ceph-mon[29131]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T17:06:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:27 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:27 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:27.359 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:27 smithi096 ceph-mon[29131]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' 2024-04-25T17:06:27.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:27 smithi097 ceph-mon[32602]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:06:27.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:27 smithi097 ceph-mon[32602]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T17:06:27.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:27 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:27.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:27 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:27.450 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:27 smithi097 ceph-mon[32602]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' 2024-04-25T17:06:27.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:27 smithi171 ceph-mon[33189]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T17:06:27.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:27 smithi171 ceph-mon[33189]: osdmap e59: 10 total, 9 up, 10 in 2024-04-25T17:06:27.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:27 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:27.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:27 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:27.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:27 smithi171 ceph-mon[33189]: from='osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194]' entity='osd.9' 2024-04-25T17:06:28.044 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T17:06:28.045 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T17:06:28.045 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T17:06:28.045 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T17:06:28.045 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130213 s, 805 MB/s 2024-04-25T17:06:28.045 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T17:06:28.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:28 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:06:28.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:28 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:06:28.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:28 smithi097 ceph-mon[32602]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:28.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:28 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:28.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:28 smithi097 ceph-mon[32602]: osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] boot 2024-04-25T17:06:28.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:28 smithi097 ceph-mon[32602]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T17:06:28.339 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:28 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:28.369 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:28 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:06:28.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:28 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:06:28.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:28 smithi096 ceph-mon[29131]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:28.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:28 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:28.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:28 smithi096 ceph-mon[29131]: osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] boot 2024-04-25T17:06:28.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:28 smithi096 ceph-mon[29131]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T17:06:28.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:28 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:28 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:06:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:28 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:06:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:28 smithi171 ceph-mon[33189]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T17:06:28.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:28 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:28.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:28 smithi171 ceph-mon[33189]: osd.9 [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] boot 2024-04-25T17:06:28.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:28 smithi171 ceph-mon[33189]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T17:06:28.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:28 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:06:28.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-04-25T17:06:29.075 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:29.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:29 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:29.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T17:06:29.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:29.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:29.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:29.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:29.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:29.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:29.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:29.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:29 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:30.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: osdmap e61: 10 total, 10 up, 10 in 2024-04-25T17:06:30.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:30.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:30.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:29 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:30.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:30 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi096 2024-04-25T17:06:30.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:30 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T17:06:30.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:30 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi096 2024-04-25T17:06:30.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:30 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T17:06:31.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:30 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi096 2024-04-25T17:06:31.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:30 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T17:06:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 -- 172.21.15.97:0/3794005875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1057a0 msgr2=0x7f1c1c105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 --2- 172.21.15.97:0/3794005875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1057a0 0x7f1c1c105bb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009990 tx=0x7f1c10009ca0 comp rx=0 tx=0).stop 2024-04-25T17:06:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 -- 172.21.15.97:0/3794005875 shutdown_connections 2024-04-25T17:06:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 --2- 172.21.15.97:0/3794005875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c1c102b40 0x7f1c1c10c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 --2- 172.21.15.97:0/3794005875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c1057a0 0x7f1c1c105bb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 --2- 172.21.15.97:0/3794005875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c1c1021f0 0x7f1c1c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 -- 172.21.15.97:0/3794005875 >> 172.21.15.97:0/3794005875 conn(0x7f1c1c0fdec0 msgr2=0x7f1c1c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 -- 172.21.15.97:0/3794005875 shutdown_connections 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 -- 172.21.15.97:0/3794005875 wait complete. 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 Processor -- start 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.057+0000 7f1c22f74700 1 -- start start 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c22f74700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c1c1021f0 0x7f1c1c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c22f74700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c102b40 0x7f1c1c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c22f74700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c1c1057a0 0x7f1c1c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c22f74700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10fc50 con 0x7f1c1c1021f0 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c22f74700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10fad0 con 0x7f1c1c102b40 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c22f74700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c10fdd0 con 0x7f1c1c1057a0 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c102b40 0x7f1c1c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c102b40 0x7f1c1c19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45916/0 (socket says 172.21.15.97:45916) 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 -- 172.21.15.97:0/692365908 learned_addr learned my addr 172.21.15.97:0/692365908 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c21511700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c1c1057a0 0x7f1c1c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c20d10700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c1c1021f0 0x7f1c1c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 -- 172.21.15.97:0/692365908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c1c1057a0 msgr2=0x7f1c1c1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c1c1057a0 0x7f1c1c1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 -- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c1c1021f0 msgr2=0x7f1c1c19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c1c1021f0 0x7f1c1c19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.058+0000 7f1c1bfff700 1 -- 172.21.15.97:0/692365908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c1a22d0 con 0x7f1c1c102b40 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.059+0000 7f1c1bfff700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c102b40 0x7f1c1c19ad40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800c9b0 tx=0x7f1c0800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.059+0000 7f1c19ffb700 1 -- 172.21.15.97:0/692365908 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08007b00 con 0x7f1c1c102b40 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.059+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c103360 con 0x7f1c1c102b40 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.059+0000 7f1c19ffb700 1 -- 172.21.15.97:0/692365908 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0800ce50 con 0x7f1c1c102b40 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.059+0000 7f1c19ffb700 1 -- 172.21.15.97:0/692365908 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08018660 con 0x7f1c1c102b40 2024-04-25T17:06:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.059+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c1038f0 con 0x7f1c1c102b40 2024-04-25T17:06:31.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.060+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1c051190 con 0x7f1c1c102b40 2024-04-25T17:06:31.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.061+0000 7f1c19ffb700 1 -- 172.21.15.97:0/692365908 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1c0800f450 con 0x7f1c1c102b40 2024-04-25T17:06:31.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.062+0000 7f1c19ffb700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1c0c0afb80 0x7f1c0c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:31.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.062+0000 7f1c19ffb700 1 -- 172.21.15.97:0/692365908 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f1c080de5d0 con 0x7f1c1c102b40 2024-04-25T17:06:31.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.062+0000 7f1c20d10700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1c0c0afb80 0x7f1c0c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:31.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.063+0000 7f1c20d10700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1c0c0afb80 0x7f1c0c0b2090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c100038a0 tx=0x7f1c1001e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:06:31.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.066+0000 7f1c19ffb700 1 -- 172.21.15.97:0/692365908 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c08014030 con 0x7f1c1c102b40 2024-04-25T17:06:31.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:31.269+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1c1c063ad0 con 0x7f1c0c0afb80 2024-04-25T17:06:31.883 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:31 smithi097 ceph-mon[32602]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:31.884 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:31.884 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:31.884 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:31.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:31 smithi096 ceph-mon[29131]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:31.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:31.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:31.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:32.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:31 smithi171 ceph-mon[33189]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:32.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:32.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:32.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:32.931 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:32 smithi171 ceph-mon[33189]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:32.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:32 smithi096 ceph-mon[29131]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:32.941 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:32 smithi097 ceph-mon[32602]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:33.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:33 smithi096 ceph-mon[29131]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:33.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:33 smithi097 ceph-mon[32602]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:34.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:33 smithi171 ceph-mon[33189]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:34 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/2047433704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]: dispatch 2024-04-25T17:06:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:34 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]: dispatch 2024-04-25T17:06:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:34 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]': finished 2024-04-25T17:06:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:34 smithi096 ceph-mon[29131]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T17:06:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:34 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/2047433704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]: dispatch 2024-04-25T17:06:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:34 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]: dispatch 2024-04-25T17:06:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:34 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]': finished 2024-04-25T17:06:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:34 smithi097 ceph-mon[32602]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T17:06:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:34 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/2047433704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]: dispatch 2024-04-25T17:06:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:34 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]: dispatch 2024-04-25T17:06:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:34 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d42cae-d4f6-4e59-b0c0-8678555c6b4b"}]': finished 2024-04-25T17:06:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:34 smithi171 ceph-mon[33189]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T17:06:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:35.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:35 smithi096 ceph-mon[29131]: from='client.? 172.21.15.97:0/4075734137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:06:35.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:35 smithi096 ceph-mon[29131]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:35.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:35 smithi097 ceph-mon[32602]: from='client.? 172.21.15.97:0/4075734137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:06:35.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:35 smithi097 ceph-mon[32602]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:36.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:35 smithi171 ceph-mon[33189]: from='client.? 172.21.15.97:0/4075734137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:06:36.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:35 smithi171 ceph-mon[33189]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:38.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:38 smithi097 ceph-mon[32602]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:38.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:38 smithi096 ceph-mon[29131]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:38 smithi171 ceph-mon[33189]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:40 smithi096 ceph-mon[29131]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:40.475 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:40 smithi097 ceph-mon[32602]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:40.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:40 smithi171 ceph-mon[33189]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:42.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:42 smithi096 ceph-mon[29131]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:42.490 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:42 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T17:06:42.491 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:42 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:42.491 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:42 smithi097 ceph-mon[32602]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:42.491 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:42 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T17:06:42.491 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:42 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:42.539 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:42 smithi171 ceph-mon[33189]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:42.539 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:42 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T17:06:42.539 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:42 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:43.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:43 smithi096 ceph-mon[29131]: Deploying daemon osd.10 on smithi097 2024-04-25T17:06:43.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:43 smithi097 ceph-mon[32602]: Deploying daemon osd.10 on smithi097 2024-04-25T17:06:43.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:43 smithi171 ceph-mon[33189]: Deploying daemon osd.10 on smithi097 2024-04-25T17:06:44.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:44 smithi096 ceph-mon[29131]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:44.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:44 smithi097 ceph-mon[32602]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:44.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:44 smithi171 ceph-mon[33189]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:46.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:46 smithi096 ceph-mon[29131]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:46 smithi097 ceph-mon[32602]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:46.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:46 smithi171 ceph-mon[33189]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:48.375 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:48 smithi097 ceph-mon[32602]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:48.375 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:48.375 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:48.375 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:48 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:48 smithi096 ceph-mon[29131]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:48.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:48 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:48 smithi171 ceph-mon[33189]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:48 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:50.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:50 smithi096 ceph-mon[29131]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:50.468 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:50 smithi097 ceph-mon[32602]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:50 smithi171 ceph-mon[33189]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:51.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.993+0000 7f1c19ffb700 1 -- 172.21.15.97:0/692365908 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1c1c063ad0 con 0x7f1c0c0afb80 2024-04-25T17:06:51.992 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 10 on host 'smithi097' 2024-04-25T17:06:51.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1c0c0afb80 msgr2=0x7f1c0c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1c0c0afb80 0x7f1c0c0b2090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c100038a0 tx=0x7f1c1001e040 comp rx=0 tx=0).stop 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c102b40 msgr2=0x7f1c1c19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c102b40 0x7f1c1c19ad40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800c9b0 tx=0x7f1c0800ccc0 comp rx=0 tx=0).stop 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 shutdown_connections 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c1c1057a0 0x7f1c1c1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1c0c0afb80 0x7f1c0c0b2090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c1c1021f0 0x7f1c1c19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 --2- 172.21.15.97:0/692365908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c1c102b40 0x7f1c1c19ad40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 >> 172.21.15.97:0/692365908 conn(0x7f1c1c0fdec0 msgr2=0x7f1c1c108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 shutdown_connections 2024-04-25T17:06:51.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T17:06:51.995+0000 7f1c22f74700 1 -- 172.21.15.97:0/692365908 wait complete. 2024-04-25T17:06:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T17:06:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T17:06:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:52 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T17:06:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T17:06:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:52 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:52 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:06:52.706 DEBUG:teuthology.orchestra.run.smithi097:osd.10> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T17:06:52.709 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2024-04-25T17:06:52.709 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T17:06:52.962 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:06:53.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-mon[32602]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T17:06:53.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-mon[32602]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T17:06:53.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:53.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-mon[32602]: from='osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:06:53.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-mon[32602]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:06:53.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-mon[32602]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:53.234 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:06:53.140+0000 7f5ca7125700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T17:06:53.234 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:06:53 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:06:53.151+0000 7f5ca2eda700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:06:53.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:53 smithi171 ceph-mon[33189]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T17:06:53.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:53 smithi171 ceph-mon[33189]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T17:06:53.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:53 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:53.351 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:53 smithi171 ceph-mon[33189]: from='osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:06:53.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:53 smithi171 ceph-mon[33189]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:06:53.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:53 smithi171 ceph-mon[33189]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:53.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:53 smithi096 ceph-mon[29131]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T17:06:53.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:53 smithi096 ceph-mon[29131]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T17:06:53.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:53 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:53.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:53 smithi096 ceph-mon[29131]: from='osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:06:53.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:53 smithi096 ceph-mon[29131]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T17:06:53.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:53 smithi096 ceph-mon[29131]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-04-25T17:06:54.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:54 smithi097 ceph-mon[32602]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:06:54.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:54 smithi097 ceph-mon[32602]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T17:06:54.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:54 smithi097 ceph-mon[32602]: osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] boot 2024-04-25T17:06:54.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:54 smithi097 ceph-mon[32602]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T17:06:54.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:54 smithi171 ceph-mon[33189]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:06:54.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:54 smithi171 ceph-mon[33189]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T17:06:54.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:54 smithi171 ceph-mon[33189]: osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] boot 2024-04-25T17:06:54.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:54 smithi171 ceph-mon[33189]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T17:06:54.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:54 smithi096 ceph-mon[29131]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T17:06:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:54 smithi096 ceph-mon[29131]: osdmap e64: 11 total, 10 up, 11 in 2024-04-25T17:06:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:54 smithi096 ceph-mon[29131]: osd.10 [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] boot 2024-04-25T17:06:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:54 smithi096 ceph-mon[29131]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T17:06:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:06:55.050 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T17:06:55.050 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T17:06:55.050 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-25T17:06:55.050 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-25T17:06:55.050 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113982 s, 920 MB/s 2024-04-25T17:06:55.050 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-25T17:06:55.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:55 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:06:55.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:55 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:06:55.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:55 smithi096 ceph-mon[29131]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T17:06:55.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:55 smithi096 ceph-mon[29131]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:55 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:06:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:55 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:06:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:55 smithi097 ceph-mon[32602]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T17:06:55.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:55 smithi097 ceph-mon[32602]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:55.773 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:55 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:06:55.773 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:55 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:06:55.773 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:55 smithi171 ceph-mon[33189]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T17:06:55.773 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:55 smithi171 ceph-mon[33189]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:55.800 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-04-25T17:06:56.053 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi097 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T17:06:56.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:56.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:56.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:56 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi097 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T17:06:56.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:56.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:56.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:56 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: osdmap e67: 11 total, 11 up, 11 in 2024-04-25T17:06:56.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi097 2024-04-25T17:06:56.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:06:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:56 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:06:57.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:57 smithi171 ceph-mon[33189]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:57.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:57 smithi096 ceph-mon[29131]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:57.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:57 smithi097 ceph-mon[32602]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 -- 172.21.15.171:0/164533207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 msgr2=0x7ff098104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 --2- 172.21.15.171:0/164533207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff098104620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009990 tx=0x7ff08c009ca0 comp rx=0 tx=0).stop 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 -- 172.21.15.171:0/164533207 shutdown_connections 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 --2- 172.21.15.171:0/164533207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098104bb0 0x7ff09810b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 --2- 172.21.15.171:0/164533207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0981030e0 0x7ff0981034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 --2- 172.21.15.171:0/164533207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff098104620 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 -- 172.21.15.171:0/164533207 >> 172.21.15.171:0/164533207 conn(0x7ff0980fe740 msgr2=0x7ff098100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 -- 172.21.15.171:0/164533207 shutdown_connections 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.782+0000 7ff09cf30700 1 -- 172.21.15.171:0/164533207 wait complete. 2024-04-25T17:06:57.783 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 Processor -- start 2024-04-25T17:06:57.784 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 -- start start 2024-04-25T17:06:57.785 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0981030e0 0x7ff09819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:57.785 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff09819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:57.785 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098104bb0 0x7ff0981a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:57.785 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff09810eb60 con 0x7ff0981030e0 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff09810e9e0 con 0x7ff098104bb0 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff09cf30700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff09810ece0 con 0x7ff098104210 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff095d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff09819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff095d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff09819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:35974/0 (socket says 172.21.15.171:35974) 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.783+0000 7ff095d9b700 1 -- 172.21.15.171:0/2954521864 learned_addr learned my addr 172.21.15.171:0/2954521864 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff09659c700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0981030e0 0x7ff09819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff096d9d700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098104bb0 0x7ff0981a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff095d9b700 1 -- 172.21.15.171:0/2954521864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098104bb0 msgr2=0x7ff0981a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff095d9b700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098104bb0 0x7ff0981a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff095d9b700 1 -- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0981030e0 msgr2=0x7ff09819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff095d9b700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0981030e0 0x7ff09819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff095d9b700 1 -- 172.21.15.171:0/2954521864 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0981a2320 con 0x7ff098104210 2024-04-25T17:06:57.786 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff095d9b700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff09819ad00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00b760 tx=0x7ff08c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:06:57.787 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff0877fe700 1 -- 172.21.15.171:0/2954521864 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c026070 con 0x7ff098104210 2024-04-25T17:06:57.787 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0981a25b0 con 0x7ff098104210 2024-04-25T17:06:57.787 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff0877fe700 1 -- 172.21.15.171:0/2954521864 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08c021410 con 0x7ff098104210 2024-04-25T17:06:57.787 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff0877fe700 1 -- 172.21.15.171:0/2954521864 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c017650 con 0x7ff098104210 2024-04-25T17:06:57.787 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.784+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0981a2a00 con 0x7ff098104210 2024-04-25T17:06:57.787 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.785+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff098051190 con 0x7ff098104210 2024-04-25T17:06:57.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.787+0000 7ff0877fe700 1 -- 172.21.15.171:0/2954521864 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff08c029070 con 0x7ff098104210 2024-04-25T17:06:57.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.787+0000 7ff0877fe700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0800afb80 0x7ff0800b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:06:57.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.787+0000 7ff0877fe700 1 -- 172.21.15.171:0/2954521864 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7ff08c0dfa90 con 0x7ff098104210 2024-04-25T17:06:57.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.787+0000 7ff09659c700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0800afb80 0x7ff0800b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:06:57.788 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.788+0000 7ff09659c700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0800afb80 0x7ff0800b2090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff098104110 tx=0x7ff088009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:06:57.791 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.791+0000 7ff0877fe700 1 -- 172.21.15.171:0/2954521864 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff08c030020 con 0x7ff098104210 2024-04-25T17:06:57.997 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:06:57.997+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff0981a2d20 con 0x7ff0800afb80 2024-04-25T17:06:58.635 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:58 smithi171 ceph-mon[33189]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:58.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:58 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:58.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:58 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:58.636 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:58 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:58.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:58 smithi096 ceph-mon[29131]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:58.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:58 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:58.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:58 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:58.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:58 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:58.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:58 smithi097 ceph-mon[32602]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:06:58.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:58 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T17:06:58.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:58 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T17:06:58.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:58 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:06:59.568 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:06:59 smithi171 ceph-mon[33189]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:59.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:06:59 smithi096 ceph-mon[29131]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:06:59.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:06:59 smithi097 ceph-mon[32602]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:01.128 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:00 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/535875294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]: dispatch 2024-04-25T17:07:01.128 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:00 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]: dispatch 2024-04-25T17:07:01.128 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:00 smithi171 ceph-mon[33189]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]': finished 2024-04-25T17:07:01.128 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:00 smithi171 ceph-mon[33189]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T17:07:01.128 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:01.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:00 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/535875294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]: dispatch 2024-04-25T17:07:01.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:00 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]: dispatch 2024-04-25T17:07:01.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:00 smithi096 ceph-mon[29131]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]': finished 2024-04-25T17:07:01.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:00 smithi096 ceph-mon[29131]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T17:07:01.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:00 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/535875294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]: dispatch 2024-04-25T17:07:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:00 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]: dispatch 2024-04-25T17:07:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:00 smithi097 ceph-mon[32602]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fec892a1-61fb-43aa-96db-30f1a3e58416"}]': finished 2024-04-25T17:07:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:00 smithi097 ceph-mon[32602]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T17:07:01.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:02.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:01 smithi097 ceph-mon[32602]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T17:07:02.135 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:01 smithi097 ceph-mon[32602]: from='client.? 172.21.15.171:0/1662944046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:07:02.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:01 smithi096 ceph-mon[29131]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T17:07:02.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:01 smithi096 ceph-mon[29131]: from='client.? 172.21.15.171:0/1662944046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:07:02.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:01 smithi171 ceph-mon[33189]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-25T17:07:02.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:01 smithi171 ceph-mon[33189]: from='client.? 172.21.15.171:0/1662944046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T17:07:04.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:04 smithi096 ceph-mon[29131]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:07:04.469 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:04 smithi171 ceph-mon[33189]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:07:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:04 smithi097 ceph-mon[32602]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T17:07:06.163 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:06 smithi171 ceph-mon[33189]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T17:07:06.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:06 smithi096 ceph-mon[29131]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T17:07:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:06 smithi097 ceph-mon[32602]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T17:07:08.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:08 smithi171 ceph-mon[33189]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:07:08.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:08 smithi096 ceph-mon[29131]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:07:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:08 smithi097 ceph-mon[32602]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:07:09.271 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T17:07:09.271 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:09 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:09.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T17:07:09.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:09 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:09.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T17:07:09.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:09 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:10.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:10 smithi096 ceph-mon[29131]: Deploying daemon osd.11 on smithi171 2024-04-25T17:07:10.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:10 smithi096 ceph-mon[29131]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:07:10.324 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:10 smithi171 ceph-mon[33189]: Deploying daemon osd.11 on smithi171 2024-04-25T17:07:10.324 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:10 smithi171 ceph-mon[33189]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:07:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:10 smithi097 ceph-mon[32602]: Deploying daemon osd.11 on smithi171 2024-04-25T17:07:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:10 smithi097 ceph-mon[32602]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T17:07:12.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:12 smithi171 ceph-mon[33189]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:12 smithi096 ceph-mon[29131]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:12 smithi097 ceph-mon[32602]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:14.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:14 smithi171 ceph-mon[33189]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:14.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:14 smithi096 ceph-mon[29131]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:14 smithi097 ceph-mon[32602]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:15.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:15.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:15.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:15 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:15.910 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:15 smithi171 ceph-mon[33189]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:15.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:15.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:15.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:15 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:15.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:15 smithi096 ceph-mon[29131]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:15 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:15 smithi097 ceph-mon[32602]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:18.389 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:18 smithi171 ceph-mon[33189]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:18.389 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.389 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.389 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:18.390 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:18.390 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:18 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:18 smithi096 ceph-mon[29131]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:18.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:18.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:18.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:18 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.470 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.470+0000 7ff0877fe700 1 -- 172.21.15.171:0/2954521864 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff0981a2d20 con 0x7ff0800afb80 2024-04-25T17:07:18.470 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2024-04-25T17:07:18.472 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0800afb80 msgr2=0x7ff0800b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:18.472 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0800afb80 0x7ff0800b2090 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff098104110 tx=0x7ff088009260 comp rx=0 tx=0).stop 2024-04-25T17:07:18.472 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 msgr2=0x7ff09819ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:18.472 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff09819ad00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00b760 tx=0x7ff08c009580 comp rx=0 tx=0).stop 2024-04-25T17:07:18.472 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 shutdown_connections 2024-04-25T17:07:18.472 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff098104bb0 0x7ff0981a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:18.472 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0800afb80 0x7ff0800b2090 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:18.473 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0981030e0 0x7ff09819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:18.473 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 --2- 172.21.15.171:0/2954521864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff098104210 0x7ff09819ad00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:18.473 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 >> 172.21.15.171:0/2954521864 conn(0x7ff0980fe740 msgr2=0x7ff0981000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:18.473 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 shutdown_connections 2024-04-25T17:07:18.473 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:18.473+0000 7ff09cf30700 1 -- 172.21.15.171:0/2954521864 wait complete. 2024-04-25T17:07:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:18 smithi097 ceph-mon[32602]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:18.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:18.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:18 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:19.179 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T17:07:19.181 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T17:07:19.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T17:07:19.432 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:07:19.555 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:19.556 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:19.556 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:19 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:19.556 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:19 smithi096 ceph-mon[29131]: from='osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T17:07:19.556 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:19 smithi096 ceph-mon[29131]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T17:07:19.556 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:19 smithi096 ceph-mon[29131]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:19.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:19 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:19.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:19 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:19.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:19 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:19.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:19 smithi171 ceph-mon[33189]: from='osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T17:07:19.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:19 smithi171 ceph-mon[33189]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T17:07:19.670 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:19 smithi171 ceph-mon[33189]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:19 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:19 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:19 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:19 smithi097 ceph-mon[32602]: from='osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T17:07:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:19 smithi097 ceph-mon[32602]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T17:07:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:19 smithi097 ceph-mon[32602]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:20.730 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:20 smithi171 ceph-mon[33189]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T17:07:20.730 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:20 smithi171 ceph-mon[33189]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T17:07:20.731 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:20 smithi171 ceph-mon[33189]: from='osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:07:20.731 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:20 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:20.731 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:20 smithi171 ceph-mon[33189]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:07:20.731 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:07:20 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:07:20.444+0000 7fe1a0bc3700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T17:07:20.731 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:07:20 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:07:20.461+0000 7fe19c978700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T17:07:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:20 smithi097 ceph-mon[32602]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T17:07:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:20 smithi097 ceph-mon[32602]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T17:07:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:20 smithi097 ceph-mon[32602]: from='osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:07:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:20 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:20 smithi097 ceph-mon[32602]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:07:20.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:20 smithi096 ceph-mon[29131]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T17:07:20.846 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:20 smithi096 ceph-mon[29131]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T17:07:20.846 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:20 smithi096 ceph-mon[29131]: from='osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:07:20.846 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:20 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:20.846 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:20 smithi096 ceph-mon[29131]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-25T17:07:21.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 -- 172.21.15.96:0/1630548025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 msgr2=0x7f6c18105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:21.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/1630548025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 0x7f6c18105bc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009990 tx=0x7f6c14009ca0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 -- 172.21.15.96:0/1630548025 shutdown_connections 2024-04-25T17:07:21.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/1630548025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c18102b50 0x7f6c1810c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/1630548025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 0x7f6c18105bc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/1630548025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18102200 0x7f6c18102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 -- 172.21.15.96:0/1630548025 >> 172.21.15.96:0/1630548025 conn(0x7f6c180fded0 msgr2=0x7f6c18100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 -- 172.21.15.96:0/1630548025 shutdown_connections 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.260+0000 7f6c1ff07700 1 -- 172.21.15.96:0/1630548025 wait complete. 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 Processor -- start 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 -- start start 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c18102200 0x7f6c1819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18102b50 0x7f6c1819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 0x7f6c181a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810fc60 con 0x7f6c181057b0 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810fae0 con 0x7f6c18102b50 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1ff07700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1810fde0 con 0x7f6c18102200 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1dca3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c18102200 0x7f6c1819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1d4a2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18102b50 0x7f6c1819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.261+0000 7f6c1e4a4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 0x7f6c181a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1dca3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c18102200 0x7f6c1819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:39642/0 (socket says 172.21.15.96:39642) 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1dca3700 1 -- 172.21.15.96:0/4212785041 learned_addr learned my addr 172.21.15.96:0/4212785041 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1e4a4700 1 -- 172.21.15.96:0/4212785041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c18102200 msgr2=0x7f6c1819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1e4a4700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c18102200 0x7f6c1819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1e4a4700 1 -- 172.21.15.96:0/4212785041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18102b50 msgr2=0x7f6c1819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1e4a4700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18102b50 0x7f6c1819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1e4a4700 1 -- 172.21.15.96:0/4212785041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c181a2300 con 0x7f6c181057b0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1e4a4700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 0x7f6c181a1c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000eb80 tx=0x7f6c1000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c0effd700 1 -- 172.21.15.96:0/4212785041 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c100092e0 con 0x7f6c181057b0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c181a2590 con 0x7f6c181057b0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c0effd700 1 -- 172.21.15.96:0/4212785041 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c1000c5f0 con 0x7f6c181057b0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.262+0000 7f6c0effd700 1 -- 172.21.15.96:0/4212785041 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10004280 con 0x7f6c181057b0 2024-04-25T17:07:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.263+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c181a2a70 con 0x7f6c181057b0 2024-04-25T17:07:21.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.264+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c18051190 con 0x7f6c181057b0 2024-04-25T17:07:21.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.265+0000 7f6c0effd700 1 -- 172.21.15.96:0/4212785041 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6c1001b030 con 0x7f6c181057b0 2024-04-25T17:07:21.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.266+0000 7f6c0effd700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6c040afb80 0x7f6c040b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:21.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.266+0000 7f6c1dca3700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6c040afb80 0x7f6c040b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:21.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.266+0000 7f6c0effd700 1 -- 172.21.15.96:0/4212785041 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6847+0+0 (secure 0 0 0) 0x7f6c100de0e0 con 0x7f6c181057b0 2024-04-25T17:07:21.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.266+0000 7f6c1dca3700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6c040afb80 0x7f6c040b2090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b2e0 tx=0x7f6c14009590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:21.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.271+0000 7f6c0effd700 1 -- 172.21.15.96:0/4212785041 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c10022020 con 0x7f6c181057b0 2024-04-25T17:07:21.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.479+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6c180642a0 con 0x7f6c181057b0 2024-04-25T17:07:21.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.479+0000 7f6c0effd700 1 -- 172.21.15.96:0/4212785041 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f6c10010040 con 0x7f6c181057b0 2024-04-25T17:07:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:21.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6c040afb80 msgr2=0x7f6c040b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6c040afb80 0x7f6c040b2090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b2e0 tx=0x7f6c14009590 comp rx=0 tx=0).stop 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 msgr2=0x7f6c181a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 0x7f6c181a1c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000eb80 tx=0x7f6c1000ee90 comp rx=0 tx=0).stop 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 shutdown_connections 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6c040afb80 0x7f6c040b2090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c181057b0 0x7f6c181a1c80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c18102200 0x7f6c1819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 --2- 172.21.15.96:0/4212785041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c18102b50 0x7f6c1819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 >> 172.21.15.96:0/4212785041 conn(0x7f6c180fded0 msgr2=0x7f6c18108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 shutdown_connections 2024-04-25T17:07:21.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:21.482+0000 7f6c1ff07700 1 -- 172.21.15.96:0/4212785041 wait complete. 2024-04-25T17:07:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:21 smithi096 ceph-mon[29131]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:07:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:21 smithi096 ceph-mon[29131]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T17:07:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:21 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:21 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:21 smithi096 ceph-mon[29131]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:21 smithi097 ceph-mon[32602]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:07:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:21 smithi097 ceph-mon[32602]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T17:07:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:21 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:21 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:21 smithi097 ceph-mon[32602]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:21.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:21 smithi171 ceph-mon[33189]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-25T17:07:21.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:21 smithi171 ceph-mon[33189]: osdmap e70: 12 total, 11 up, 12 in 2024-04-25T17:07:21.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:21 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:21.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:21 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:21.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:21 smithi171 ceph-mon[33189]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-25T17:07:22.163 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1714064841,"num_in_osds":12,"osd_in_since":1714064820,"num_remapped_pgs":0} 2024-04-25T17:07:22.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T17:07:22.413 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: purged_snaps scrub starts 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: purged_snaps scrub ok 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] boot 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/4212785041' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:22.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:22.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:22 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: purged_snaps scrub starts 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: purged_snaps scrub ok 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] boot 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/4212785041' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:22 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: purged_snaps scrub starts 2024-04-25T17:07:22.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: purged_snaps scrub ok 2024-04-25T17:07:22.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: osd.11 [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] boot 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/4212785041' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:22 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:23.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:23 smithi096 ceph-mon[29131]: Detected new or changed devices on smithi171 2024-04-25T17:07:23.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:23 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-25T17:07:23.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:23 smithi096 ceph-mon[29131]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T17:07:23.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:23 smithi096 ceph-mon[29131]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:23 smithi097 ceph-mon[32602]: Detected new or changed devices on smithi171 2024-04-25T17:07:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:23 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-25T17:07:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:23 smithi097 ceph-mon[32602]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T17:07:23.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:23 smithi097 ceph-mon[32602]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:23.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:23 smithi171 ceph-mon[33189]: Detected new or changed devices on smithi171 2024-04-25T17:07:23.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:23 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-25T17:07:23.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:23 smithi171 ceph-mon[33189]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T17:07:23.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:23 smithi171 ceph-mon[33189]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:24.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/46551854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 msgr2=0x7f8fd8104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/46551854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd8104870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80099c0 tx=0x7f8fc8009cd0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/46551854 shutdown_connections 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/46551854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd81096e0 0x7f8fd8111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/46551854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd8104870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/46551854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd8108d90 0x7f8fd81091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/46551854 >> 172.21.15.96:0/46551854 conn(0x7f8fd80ffd40 msgr2=0x7f8fd8102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/46551854 shutdown_connections 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.244+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/46551854 wait complete. 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 Processor -- start 2024-04-25T17:07:24.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 -- start start 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd8108d90 0x7f8fd81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd81096e0 0x7f8fd819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8114330 con 0x7f8fd8104460 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81141b0 con 0x7f8fd81096e0 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fdd6b6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81144b0 con 0x7f8fd8108d90 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fd6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fd6ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd81a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36868/0 (socket says 172.21.15.96:36868) 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fd6ffd700 1 -- 172.21.15.96:0/3988069353 learned_addr learned my addr 172.21.15.96:0/3988069353 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fd67fc700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd8108d90 0x7f8fd81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.245+0000 7f8fd77fe700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd81096e0 0x7f8fd819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fd6ffd700 1 -- 172.21.15.96:0/3988069353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd8108d90 msgr2=0x7f8fd81a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fd6ffd700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd8108d90 0x7f8fd81a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fd6ffd700 1 -- 172.21.15.96:0/3988069353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd81096e0 msgr2=0x7f8fd819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fd6ffd700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd81096e0 0x7f8fd819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fd6ffd700 1 -- 172.21.15.96:0/3988069353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd819b160 con 0x7f8fd8104460 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fd77fe700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd81096e0 0x7f8fd819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fd6ffd700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd81a0890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b580 tx=0x7f8fc80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fbffff700 1 -- 172.21.15.96:0/3988069353 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8027070 con 0x7f8fd8104460 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd819b3f0 con 0x7f8fd8104460 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fbffff700 1 -- 172.21.15.96:0/3988069353 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc8003f20 con 0x7f8fd8104460 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fbffff700 1 -- 172.21.15.96:0/3988069353 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8022d70 con 0x7f8fd8104460 2024-04-25T17:07:24.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.246+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd819b8d0 con 0x7f8fd8104460 2024-04-25T17:07:24.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.247+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd8051190 con 0x7f8fd8104460 2024-04-25T17:07:24.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.248+0000 7f8fbffff700 1 -- 172.21.15.96:0/3988069353 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8fc8022500 con 0x7f8fd8104460 2024-04-25T17:07:24.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.249+0000 7f8fbffff700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fc00afb80 0x7f8fc00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:24.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.249+0000 7f8fd67fc700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fc00afb80 0x7f8fc00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:24.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.249+0000 7f8fbffff700 1 -- 172.21.15.96:0/3988069353 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8fc80df790 con 0x7f8fd8104460 2024-04-25T17:07:24.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.249+0000 7f8fd67fc700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fc00afb80 0x7f8fc00b2090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8108cf0 tx=0x7f8fcc00a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:24.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.253+0000 7f8fbffff700 1 -- 172.21.15.96:0/3988069353 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fc8013070 con 0x7f8fd8104460 2024-04-25T17:07:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.453+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fd802e510 con 0x7f8fd8104460 2024-04-25T17:07:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.454+0000 7f8fbffff700 1 -- 172.21.15.96:0/3988069353 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18408 (secure 0 0 0) 0x7f8fc80a88a0 con 0x7f8fd8104460 2024-04-25T17:07:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:24.454 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":72,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:07:22.448056+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fc00afb80 msgr2=0x7f8fc00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fc00afb80 0x7f8fc00b2090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8108cf0 tx=0x7f8fcc00a300 comp rx=0 tx=0).stop 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 msgr2=0x7f8fd81a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd81a0890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b580 tx=0x7f8fc80047b0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 shutdown_connections 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd81096e0 0x7f8fd819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fc00afb80 0x7f8fc00b2090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fd8104460 0x7f8fd81a0890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 --2- 172.21.15.96:0/3988069353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd8108d90 0x7f8fd81a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.456+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 >> 172.21.15.96:0/3988069353 conn(0x7f8fd80ffd40 msgr2=0x7f8fd8102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.457+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 shutdown_connections 2024-04-25T17:07:24.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:24.457+0000 7f8fdd6b6700 1 -- 172.21.15.96:0/3988069353 wait complete. 2024-04-25T17:07:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:24 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3988069353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:07:24.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:24 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3988069353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:07:24.882 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:24 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3988069353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:07:25.296 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:07:25.296 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T17:07:25.534 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:07:25.657 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:25 smithi096 ceph-mon[29131]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:25.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:25 smithi097 ceph-mon[32602]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:25.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:25 smithi171 ceph-mon[33189]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 -- 172.21.15.96:0/4014444556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc81029f0 msgr2=0x7fbcc8102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 --2- 172.21.15.96:0/4014444556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc81029f0 0x7fbcc8102e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40099c0 tx=0x7fbcc4009cd0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 -- 172.21.15.96:0/4014444556 shutdown_connections 2024-04-25T17:07:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 --2- 172.21.15.96:0/4014444556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc8109760 0x7fbcc81110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 --2- 172.21.15.96:0/4014444556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc81029f0 0x7fbcc8102e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 --2- 172.21.15.96:0/4014444556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcc8108e10 0x7fbcc8109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 -- 172.21.15.96:0/4014444556 >> 172.21.15.96:0/4014444556 conn(0x7fbcc80fe3f0 msgr2=0x7fbcc8100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 -- 172.21.15.96:0/4014444556 shutdown_connections 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.337+0000 7fbcd066e700 1 -- 172.21.15.96:0/4014444556 wait complete. 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 Processor -- start 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 -- start start 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcc81029f0 0x7fbcc819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc8108e10 0x7fbcc819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc8109760 0x7fbcc819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc81144d0 con 0x7fbcc8109760 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8114350 con 0x7fbcc8108e10 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbcd066e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc8114650 con 0x7fbcc81029f0 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbccec0b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc8109760 0x7fbcc819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbccec0b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc8109760 0x7fbcc819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36884/0 (socket says 172.21.15.96:36884) 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.338+0000 7fbccec0b700 1 -- 172.21.15.96:0/1940597528 learned_addr learned my addr 172.21.15.96:0/1940597528 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbccdc09700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc8108e10 0x7fbcc819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbcce40a700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcc81029f0 0x7fbcc819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbccec0b700 1 -- 172.21.15.96:0/1940597528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcc81029f0 msgr2=0x7fbcc819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbccec0b700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcc81029f0 0x7fbcc819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbccec0b700 1 -- 172.21.15.96:0/1940597528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc8108e10 msgr2=0x7fbcc819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbccec0b700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc8108e10 0x7fbcc819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbccec0b700 1 -- 172.21.15.96:0/1940597528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc819fcf0 con 0x7fbcc8109760 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbccec0b700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc8109760 0x7fbcc819f650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000b970 tx=0x7fbcc000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbcbf7fe700 1 -- 172.21.15.96:0/1940597528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0019070 con 0x7fbcc8109760 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbcbf7fe700 1 -- 172.21.15.96:0/1940597528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc000cde0 con 0x7fbcc8109760 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc819ffe0 con 0x7fbcc8109760 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbcbf7fe700 1 -- 172.21.15.96:0/1940597528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0007370 con 0x7fbcc8109760 2024-04-25T17:07:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.339+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc81a7900 con 0x7fbcc8109760 2024-04-25T17:07:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.340+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc810c740 con 0x7fbcc8109760 2024-04-25T17:07:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.342+0000 7fbcbf7fe700 1 -- 172.21.15.96:0/1940597528 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbcc001c070 con 0x7fbcc8109760 2024-04-25T17:07:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.343+0000 7fbcbf7fe700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcb40afb80 0x7fbcb40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.343+0000 7fbcce40a700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcb40afb80 0x7fbcb40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.343+0000 7fbcbf7fe700 1 -- 172.21.15.96:0/1940597528 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbcc00de3a0 con 0x7fbcc8109760 2024-04-25T17:07:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.343+0000 7fbcce40a700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcb40afb80 0x7fbcb40b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4000c00 tx=0x7fbcc401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:27.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.347+0000 7fbcbf7fe700 1 -- 172.21.15.96:0/1940597528 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcc0023070 con 0x7fbcc8109760 2024-04-25T17:07:27.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.543+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbcc819c180 con 0x7fbcc8109760 2024-04-25T17:07:27.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.543+0000 7fbcbf7fe700 1 -- 172.21.15.96:0/1940597528 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fbcc00a7420 con 0x7fbcc8109760 2024-04-25T17:07:27.543 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcb40afb80 msgr2=0x7fbcb40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcb40afb80 0x7fbcb40b2090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4000c00 tx=0x7fbcc401a040 comp rx=0 tx=0).stop 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc8109760 msgr2=0x7fbcc819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc8109760 0x7fbcc819f650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000b970 tx=0x7fbcc000bc80 comp rx=0 tx=0).stop 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 shutdown_connections 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcb40afb80 0x7fbcb40b2090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbcc8109760 0x7fbcc819f650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbcc81029f0 0x7fbcc819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 --2- 172.21.15.96:0/1940597528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbcc8108e10 0x7fbcc819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:27.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 >> 172.21.15.96:0/1940597528 conn(0x7fbcc80fe3f0 msgr2=0x7fbcc8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:27.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 shutdown_connections 2024-04-25T17:07:27.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:27.545+0000 7fbcd066e700 1 -- 172.21.15.96:0/1940597528 wait complete. 2024-04-25T17:07:28.296 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:28 smithi096 ceph-mon[29131]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:28.296 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:28 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1940597528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:07:28.322 INFO:tasks.cephadm:Adding mds.a on smithi096 2024-04-25T17:07:28.322 INFO:tasks.cephadm:Adding mds.d on smithi096 2024-04-25T17:07:28.322 INFO:tasks.cephadm:Adding mds.g on smithi096 2024-04-25T17:07:28.322 INFO:tasks.cephadm:Adding mds.j on smithi096 2024-04-25T17:07:28.322 INFO:tasks.cephadm:Adding mds.b on smithi097 2024-04-25T17:07:28.322 INFO:tasks.cephadm:Adding mds.e on smithi097 2024-04-25T17:07:28.323 INFO:tasks.cephadm:Adding mds.h on smithi097 2024-04-25T17:07:28.323 INFO:tasks.cephadm:Adding mds.k on smithi097 2024-04-25T17:07:28.323 INFO:tasks.cephadm:Adding mds.c on smithi171 2024-04-25T17:07:28.323 INFO:tasks.cephadm:Adding mds.f on smithi171 2024-04-25T17:07:28.323 INFO:tasks.cephadm:Adding mds.i on smithi171 2024-04-25T17:07:28.323 INFO:tasks.cephadm:Adding mds.l on smithi171 2024-04-25T17:07:28.323 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2024-04-25T17:07:28.350 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:28 smithi171 ceph-mon[33189]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:28.350 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:28 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1940597528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:07:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:28 smithi097 ceph-mon[32602]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:28 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1940597528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:07:28.574 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:07:30.199 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 -- 172.21.15.171:0/403024516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc1057b0 msgr2=0x7f0adc105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:30.199 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 --2- 172.21.15.171:0/403024516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc1057b0 0x7f0adc105bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009990 tx=0x7f0acc009ca0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 -- 172.21.15.171:0/403024516 shutdown_connections 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 --2- 172.21.15.171:0/403024516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0adc102b50 0x7f0adc10c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 --2- 172.21.15.171:0/403024516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc1057b0 0x7f0adc105bc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 --2- 172.21.15.171:0/403024516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc102200 0x7f0adc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 -- 172.21.15.171:0/403024516 >> 172.21.15.171:0/403024516 conn(0x7f0adc0fded0 msgr2=0x7f0adc100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 -- 172.21.15.171:0/403024516 shutdown_connections 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.198+0000 7f0ae2990700 1 -- 172.21.15.171:0/403024516 wait complete. 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 Processor -- start 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 -- start start 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc102200 0x7f0adc104d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc102b50 0x7f0adc103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0adc1057b0 0x7f0adc103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:30.200 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc10fca0 con 0x7f0adc1057b0 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc10fb20 con 0x7f0adc102200 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0ae2990700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc10fe20 con 0x7f0adc102b50 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0adb7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc102b50 0x7f0adc103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0adb7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc102b50 0x7f0adc103430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:39044/0 (socket says 172.21.15.171:39044) 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0adb7fe700 1 -- 172.21.15.171:0/1569908033 learned_addr learned my addr 172.21.15.171:0/1569908033 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.199+0000 7f0adbfff700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc102200 0x7f0adc104d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0ae0f2d700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0adc1057b0 0x7f0adc103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0adb7fe700 1 -- 172.21.15.171:0/1569908033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc102200 msgr2=0x7f0adc104d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0adb7fe700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc102200 0x7f0adc104d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0adb7fe700 1 -- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0adc1057b0 msgr2=0x7f0adc103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0adb7fe700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0adc1057b0 0x7f0adc103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0adb7fe700 1 -- 172.21.15.171:0/1569908033 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc1040d0 con 0x7f0adc102b50 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0adb7fe700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc102b50 0x7f0adc103430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d950 tx=0x7f0ad000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0ad97fa700 1 -- 172.21.15.171:0/1569908033 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0016070 con 0x7f0adc102b50 2024-04-25T17:07:30.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0ad97fa700 1 -- 172.21.15.171:0/1569908033 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad0009df0 con 0x7f0adc102b50 2024-04-25T17:07:30.202 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0ad97fa700 1 -- 172.21.15.171:0/1569908033 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000f3c0 con 0x7f0adc102b50 2024-04-25T17:07:30.202 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc1a48b0 con 0x7f0adc102b50 2024-04-25T17:07:30.202 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.200+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc1a4d40 con 0x7f0adc102b50 2024-04-25T17:07:30.202 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.201+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc051190 con 0x7f0adc102b50 2024-04-25T17:07:30.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.202+0000 7f0ad97fa700 1 -- 172.21.15.171:0/1569908033 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0ad0019070 con 0x7f0adc102b50 2024-04-25T17:07:30.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.203+0000 7f0ad97fa700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0ac40afb80 0x7f0ac40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:30.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.203+0000 7f0ad97fa700 1 -- 172.21.15.171:0/1569908033 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0ad00de360 con 0x7f0adc102b50 2024-04-25T17:07:30.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.203+0000 7f0adbfff700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0ac40afb80 0x7f0ac40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:30.203 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.204+0000 7f0adbfff700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0ac40afb80 0x7f0ac40b2090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0038a0 tx=0x7f0acc01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:30.207 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.208+0000 7f0ad97fa700 1 -- 172.21.15.171:0/1569908033 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ad00a73e0 con 0x7f0adc102b50 2024-04-25T17:07:30.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:30 smithi171 ceph-mon[33189]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:30.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.423+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) -- 0x7f0adc063d90 con 0x7f0ac40afb80 2024-04-25T17:07:30.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.432+0000 7f0ad97fa700 1 -- 172.21.15.171:0/1569908033 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0adc063d90 con 0x7f0ac40afb80 2024-04-25T17:07:30.431 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2024-04-25T17:07:30.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0ac40afb80 msgr2=0x7f0ac40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:30.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0ac40afb80 0x7f0ac40b2090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0038a0 tx=0x7f0acc01a040 comp rx=0 tx=0).stop 2024-04-25T17:07:30.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc102b50 msgr2=0x7f0adc103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:30.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc102b50 0x7f0adc103430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d950 tx=0x7f0ad000dc60 comp rx=0 tx=0).stop 2024-04-25T17:07:30.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 shutdown_connections 2024-04-25T17:07:30.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0ac40afb80 0x7f0ac40b2090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.434 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0adc1057b0 0x7f0adc103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.434 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0adc102200 0x7f0adc104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.434 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 --2- 172.21.15.171:0/1569908033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0adc102b50 0x7f0adc103430 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:30.434 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 >> 172.21.15.171:0/1569908033 conn(0x7f0adc0fded0 msgr2=0x7f0adc108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:30.434 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 shutdown_connections 2024-04-25T17:07:30.434 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-25T17:07:30.434+0000 7f0ae2990700 1 -- 172.21.15.171:0/1569908033 wait complete. 2024-04-25T17:07:30.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:30 smithi096 ceph-mon[29131]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:30 smithi097 ceph-mon[32602]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:31.067 DEBUG:teuthology.orchestra.run.smithi096:mds.a> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T17:07:31.070 DEBUG:teuthology.orchestra.run.smithi096:mds.d> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T17:07:31.113 DEBUG:teuthology.orchestra.run.smithi096:mds.g> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T17:07:31.156 DEBUG:teuthology.orchestra.run.smithi096:mds.j> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T17:07:31.199 DEBUG:teuthology.orchestra.run.smithi097:mds.b> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T17:07:31.201 DEBUG:teuthology.orchestra.run.smithi097:mds.e> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T17:07:31.244 DEBUG:teuthology.orchestra.run.smithi097:mds.h> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T17:07:31.288 DEBUG:teuthology.orchestra.run.smithi097:mds.k> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T17:07:31.331 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T17:07:31.333 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T17:07:31.377 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T17:07:31.420 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T17:07:31.463 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T17:07:31.463 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:07:31.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:07:31.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:31.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:31 smithi097 ceph-mon[32602]: Deploying daemon mds.l on smithi171 2024-04-25T17:07:31.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 -- 172.21.15.96:0/1244174317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94105080 msgr2=0x7fea94105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 --2- 172.21.15.96:0/1244174317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94105080 0x7fea94105490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fea90009ca0 tx=0x7fea9002a8c0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 -- 172.21.15.96:0/1244174317 shutdown_connections 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 --2- 172.21.15.96:0/1244174317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea94102230 0x7fea9410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 --2- 172.21.15.96:0/1244174317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea94101730 0x7fea94101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 --2- 172.21.15.96:0/1244174317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94105080 0x7fea94105490 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 -- 172.21.15.96:0/1244174317 >> 172.21.15.96:0/1244174317 conn(0x7fea940fd330 msgr2=0x7fea940ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 -- 172.21.15.96:0/1244174317 shutdown_connections 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.584+0000 7fea9ba30640 1 -- 172.21.15.96:0/1244174317 wait complete. 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 Processor -- start 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 -- start start 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea94101730 0x7fea9419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94102230 0x7fea9419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea94105080 0x7fea941a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410f5f0 con 0x7fea94102230 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410f470 con 0x7fea94101730 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea9ba30640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea9410f770 con 0x7fea94105080 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94102230 0x7fea9419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94102230 0x7fea9419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35358/0 (socket says 172.21.15.96:35358) 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 -- 172.21.15.96:0/2378375052 learned_addr learned my addr 172.21.15.96:0/2378375052 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea99fcd640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea94105080 0x7fea941a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea997cc640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea94101730 0x7fea9419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 -- 172.21.15.96:0/2378375052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea94105080 msgr2=0x7fea941a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea94105080 0x7fea941a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 -- 172.21.15.96:0/2378375052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea94101730 msgr2=0x7fea9419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea94101730 0x7fea9419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.585+0000 7fea98fcb640 1 -- 172.21.15.96:0/2378375052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea941a1d90 con 0x7fea94102230 2024-04-25T17:07:31.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.586+0000 7fea98fcb640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94102230 0x7fea9419a790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea8400d950 tx=0x7fea8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:31.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.586+0000 7fea8a7fc640 1 -- 172.21.15.96:0/2378375052 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84017030 con 0x7fea94102230 2024-04-25T17:07:31.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.586+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea941a2080 con 0x7fea94102230 2024-04-25T17:07:31.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.586+0000 7fea8a7fc640 1 -- 172.21.15.96:0/2378375052 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea84004020 con 0x7fea94102230 2024-04-25T17:07:31.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.586+0000 7fea8a7fc640 1 -- 172.21.15.96:0/2378375052 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea840047a0 con 0x7fea94102230 2024-04-25T17:07:31.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.586+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea941a25d0 con 0x7fea94102230 2024-04-25T17:07:31.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.587+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea94108a10 con 0x7fea94102230 2024-04-25T17:07:31.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.589+0000 7fea8a7fc640 1 -- 172.21.15.96:0/2378375052 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fea84009a30 con 0x7fea94102230 2024-04-25T17:07:31.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.590+0000 7fea8a7fc640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fea800afaf0 0x7fea800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.590+0000 7fea997cc640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fea800afaf0 0x7fea800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.590+0000 7fea8a7fc640 1 -- 172.21.15.96:0/2378375052 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fea840de820 con 0x7fea94102230 2024-04-25T17:07:31.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.590+0000 7fea997cc640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fea800afaf0 0x7fea800b2000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea90000f80 tx=0x7fea90004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:31.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.594+0000 7fea8a7fc640 1 -- 172.21.15.96:0/2378375052 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea840a78a0 con 0x7fea94102230 2024-04-25T17:07:31.678 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:31.679 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:31 smithi171 ceph-mon[33189]: Deploying daemon mds.l on smithi171 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:31.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:31 smithi096 ceph-mon[29131]: Deploying daemon mds.l on smithi171 2024-04-25T17:07:31.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.773+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea9402dd20 con 0x7fea94102230 2024-04-25T17:07:31.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.774+0000 7fea8a7fc640 1 -- 172.21.15.96:0/2378375052 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18408 (secure 0 0 0) 0x7fea840a71a0 con 0x7fea94102230 2024-04-25T17:07:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:31.774 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":72,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:07:22.448056+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fea800afaf0 msgr2=0x7fea800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fea800afaf0 0x7fea800b2000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fea90000f80 tx=0x7fea90004a40 comp rx=0 tx=0).stop 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94102230 msgr2=0x7fea9419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94102230 0x7fea9419a790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea8400d950 tx=0x7fea8400de10 comp rx=0 tx=0).stop 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 shutdown_connections 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fea800afaf0 0x7fea800b2000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea94105080 0x7fea941a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea94102230 0x7fea9419a790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 --2- 172.21.15.96:0/2378375052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea94101730 0x7fea9419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 >> 172.21.15.96:0/2378375052 conn(0x7fea940fd330 msgr2=0x7fea94105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 shutdown_connections 2024-04-25T17:07:31.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.776+0000 7fea9ba30640 1 -- 172.21.15.96:0/2378375052 wait complete. 2024-04-25T17:07:31.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:07:31.788 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/1445791583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f181050a0 msgr2=0x7f0f181054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/1445791583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f181050a0 0x7f0f181054b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009900 tx=0x7f0f0002a410 comp rx=0 tx=0).stop 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/1445791583 shutdown_connections 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/1445791583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f18102250 0x7f0f1810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/1445791583 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f18101750 0x7f0f18101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/1445791583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f181050a0 0x7f0f181054b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/1445791583 >> 172.21.15.96:0/1445791583 conn(0x7f0f180fd350 msgr2=0x7f0f180ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/1445791583 shutdown_connections 2024-04-25T17:07:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/1445791583 wait complete. 2024-04-25T17:07:31.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 Processor -- start 2024-04-25T17:07:31.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.899+0000 7f0f1e6b6640 1 -- start start 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f1e6b6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f18101750 0x7f0f18104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f1e6b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f18102250 0x7f0f181028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f1e6b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f181050a0 0x7f0f18102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f1e6b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810f5f0 con 0x7f0f18102250 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f1e6b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810f470 con 0x7f0f181050a0 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f1e6b6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810f770 con 0x7f0f18101750 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f18102250 0x7f0f181028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f18102250 0x7f0f181028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35374/0 (socket says 172.21.15.96:35374) 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 -- 172.21.15.96:0/2933355708 learned_addr learned my addr 172.21.15.96:0/2933355708 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f1cc53640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f181050a0 0x7f0f18102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f17fff640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f18101750 0x7f0f18104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 -- 172.21.15.96:0/2933355708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f18101750 msgr2=0x7f0f18104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f18101750 0x7f0f18104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 -- 172.21.15.96:0/2933355708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f181050a0 msgr2=0x7f0f18102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f181050a0 0x7f0f18102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 -- 172.21.15.96:0/2933355708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f18103560 con 0x7f0f18102250 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f17fff640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f18101750 0x7f0f18104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.900+0000 7f0f177fe640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f18102250 0x7f0f181028f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800d950 tx=0x7f0f0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.901+0000 7f0f157fa640 1 -- 172.21.15.96:0/2933355708 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08017070 con 0x7f0f18102250 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.901+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a42e0 con 0x7f0f18102250 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.901+0000 7f0f157fa640 1 -- 172.21.15.96:0/2933355708 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f08009c30 con 0x7f0f18102250 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.901+0000 7f0f157fa640 1 -- 172.21.15.96:0/2933355708 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0800f620 con 0x7f0f18102250 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.901+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a47a0 con 0x7f0f18102250 2024-04-25T17:07:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.902+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18108a30 con 0x7f0f18102250 2024-04-25T17:07:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.903+0000 7f0f157fa640 1 -- 172.21.15.96:0/2933355708 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0f08010460 con 0x7f0f18102250 2024-04-25T17:07:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.904+0000 7f0f157fa640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0f040afaf0 0x7f0f040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.904+0000 7f0f17fff640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0f040afaf0 0x7f0f040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.904+0000 7f0f157fa640 1 -- 172.21.15.96:0/2933355708 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0f080de380 con 0x7f0f18102250 2024-04-25T17:07:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.904+0000 7f0f17fff640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0f040afaf0 0x7f0f040b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005f70 tx=0x7f0f00000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:31.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:31.908+0000 7f0f157fa640 1 -- 172.21.15.96:0/2933355708 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f080a73e0 con 0x7f0f18102250 2024-04-25T17:07:32.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.085+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0f180637a0 con 0x7f0f18102250 2024-04-25T17:07:32.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.086+0000 7f0f157fa640 1 -- 172.21.15.96:0/2933355708 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f0f08009de0 con 0x7f0f18102250 2024-04-25T17:07:32.085 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:07:32.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0f040afaf0 msgr2=0x7f0f040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:32.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0f040afaf0 0x7f0f040b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005f70 tx=0x7f0f00000bc0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f18102250 msgr2=0x7f0f181028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f18102250 0x7f0f181028f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800d950 tx=0x7f0f0800de10 comp rx=0 tx=0).stop 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 shutdown_connections 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f181050a0 0x7f0f18102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f18102250 0x7f0f181028f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0f040afaf0 0x7f0f040b2000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 --2- 172.21.15.96:0/2933355708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f18101750 0x7f0f18104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 >> 172.21.15.96:0/2933355708 conn(0x7f0f180fd350 msgr2=0x7f0f18105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 shutdown_connections 2024-04-25T17:07:32.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.088+0000 7f0f1e6b6640 1 -- 172.21.15.96:0/2933355708 wait complete. 2024-04-25T17:07:32.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 -- 172.21.15.96:0/1496134472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8101750 msgr2=0x7f17c8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 --2- 172.21.15.96:0/1496134472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8101750 0x7f17c8101bc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009900 tx=0x7f17b402a410 comp rx=0 tx=0).stop 2024-04-25T17:07:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 -- 172.21.15.96:0/1496134472 shutdown_connections 2024-04-25T17:07:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 --2- 172.21.15.96:0/1496134472 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c8102250 0x7f17c810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 --2- 172.21.15.96:0/1496134472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c8101750 0x7f17c8101bc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 --2- 172.21.15.96:0/1496134472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c81050a0 0x7f17c81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 -- 172.21.15.96:0/1496134472 >> 172.21.15.96:0/1496134472 conn(0x7f17c80fd370 msgr2=0x7f17c80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 -- 172.21.15.96:0/1496134472 shutdown_connections 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.207+0000 7f17cec34640 1 -- 172.21.15.96:0/1496134472 wait complete. 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 Processor -- start 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 -- start start 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c8101750 0x7f17c819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c8102250 0x7f17c819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81050a0 0x7f17c81a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17c810f610 con 0x7f17c81050a0 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17c810f490 con 0x7f17c8102250 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cec34640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17c810f790 con 0x7f17c8101750 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81050a0 0x7f17c81a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81050a0 0x7f17c81a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35402/0 (socket says 172.21.15.96:35402) 2024-04-25T17:07:32.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 -- 172.21.15.96:0/3115993147 learned_addr learned my addr 172.21.15.96:0/3115993147 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cc9d0640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c8101750 0x7f17c819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17c7fff640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c8102250 0x7f17c819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 -- 172.21.15.96:0/3115993147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c8101750 msgr2=0x7f17c819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c8101750 0x7f17c819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 -- 172.21.15.96:0/3115993147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c8102250 msgr2=0x7f17c819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c8102250 0x7f17c819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cd1d1640 1 -- 172.21.15.96:0/3115993147 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c81a1d20 con 0x7f17c81050a0 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.208+0000 7f17cc9d0640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c8101750 0x7f17c819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.209+0000 7f17cd1d1640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81050a0 0x7f17c81a16b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009fd0 tx=0x7f17c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.209+0000 7f17c5ffb640 1 -- 172.21.15.96:0/3115993147 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0017070 con 0x7f17c81050a0 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.209+0000 7f17c5ffb640 1 -- 172.21.15.96:0/3115993147 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c0003e20 con 0x7f17c81050a0 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.209+0000 7f17c5ffb640 1 -- 172.21.15.96:0/3115993147 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0010460 con 0x7f17c81050a0 2024-04-25T17:07:32.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.209+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c81a1fb0 con 0x7f17c81050a0 2024-04-25T17:07:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.209+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c81a2470 con 0x7f17c81050a0 2024-04-25T17:07:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.210+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c8108a30 con 0x7f17c81050a0 2024-04-25T17:07:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.211+0000 7f17c5ffb640 1 -- 172.21.15.96:0/3115993147 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f17c0003970 con 0x7f17c81050a0 2024-04-25T17:07:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.212+0000 7f17c5ffb640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17b80afb70 0x7f17b80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.212+0000 7f17cc9d0640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17b80afb70 0x7f17b80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.212+0000 7f17c5ffb640 1 -- 172.21.15.96:0/3115993147 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f17c00de5b0 con 0x7f17c81050a0 2024-04-25T17:07:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.212+0000 7f17cc9d0640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17b80afb70 0x7f17b80b2080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17c806b320 tx=0x7f17bc005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.216+0000 7f17c5ffb640 1 -- 172.21.15.96:0/3115993147 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17c00a7630 con 0x7f17c81050a0 2024-04-25T17:07:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.430+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17c802db50 con 0x7f17c81050a0 2024-04-25T17:07:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.433+0000 7f17c5ffb640 1 -- 172.21.15.96:0/3115993147 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f17c00a6f30 con 0x7f17c81050a0 2024-04-25T17:07:32.433 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:32.433 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T17:07:32.433 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 1 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17b80afb70 msgr2=0x7f17b80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17b80afb70 0x7f17b80b2080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17c806b320 tx=0x7f17bc005e50 comp rx=0 tx=0).stop 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81050a0 msgr2=0x7f17c81a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81050a0 0x7f17c81a16b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009fd0 tx=0x7f17c000b260 comp rx=0 tx=0).stop 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 shutdown_connections 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17c81050a0 0x7f17c81a16b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c8102250 0x7f17c819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17b80afb70 0x7f17b80b2080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 --2- 172.21.15.96:0/3115993147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c8101750 0x7f17c819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:32.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 >> 172.21.15.96:0/3115993147 conn(0x7f17c80fd370 msgr2=0x7f17c8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:32.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 shutdown_connections 2024-04-25T17:07:32.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:32.435+0000 7f17cec34640 1 -- 172.21.15.96:0/3115993147 wait complete. 2024-04-25T17:07:32.631 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:32 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2378375052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:07:32.631 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:32 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2933355708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:07:32.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:32 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2378375052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:07:32.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:32 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2933355708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:07:32.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:32 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2378375052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:07:32.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:32 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2933355708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:07:33.280 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 systemd[1]: Starting Ceph mds.l for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:33.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:33 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3115993147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:33.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:33 smithi171 ceph-mon[33189]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:33.563 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 podman[64025]: 2024-04-25 17:07:33.189864593 +0000 UTC m=+0.018302018 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:33.563 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 podman[64025]: 2024-04-25 17:07:33.308701708 +0000 UTC m=+0.137139174 container create fac5ba1796bceced292879c4ceb4cc2580ba3ce5f1f8dd0ef495de0b686507fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T17:07:33.563 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 podman[64025]: 2024-04-25 17:07:33.528336654 +0000 UTC m=+0.356774064 container init fac5ba1796bceced292879c4ceb4cc2580ba3ce5f1f8dd0ef495de0b686507fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T17:07:33.563 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 podman[64025]: 2024-04-25 17:07:33.531004052 +0000 UTC m=+0.359441463 container start fac5ba1796bceced292879c4ceb4cc2580ba3ce5f1f8dd0ef495de0b686507fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T17:07:33.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:33 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3115993147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:33.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:33 smithi097 ceph-mon[32602]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:33.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:33 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3115993147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:33.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:33 smithi096 ceph-mon[29131]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:34.030 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 ceph-mds[64044]: -- 172.21.15.171:0/4021960072 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fe781da40 con 0x563fe7865180 2024-04-25T17:07:34.030 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l[64040]: starting mds.l at 2024-04-25T17:07:34.030 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 bash[64025]: fac5ba1796bceced292879c4ceb4cc2580ba3ce5f1f8dd0ef495de0b686507fa 2024-04-25T17:07:34.030 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:07:33 smithi171 systemd[1]: Started Ceph mds.l for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:34.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 -- 172.21.15.96:0/3122284866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10106c10 msgr2=0x7fca101090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 --2- 172.21.15.96:0/3122284866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10106c10 0x7fca101090f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca04009ca0 tx=0x7fca0402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 -- 172.21.15.96:0/3122284866 shutdown_connections 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 --2- 172.21.15.96:0/3122284866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10109780 0x7fca1010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 --2- 172.21.15.96:0/3122284866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10106c10 0x7fca101090f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 --2- 172.21.15.96:0/3122284866 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca10101e80 0x7fca10102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 -- 172.21.15.96:0/3122284866 >> 172.21.15.96:0/3122284866 conn(0x7fca100fd330 msgr2=0x7fca100ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 -- 172.21.15.96:0/3122284866 shutdown_connections 2024-04-25T17:07:34.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.568+0000 7fca18306640 1 -- 172.21.15.96:0/3122284866 wait complete. 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 Processor -- start 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 -- start start 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca10101e80 0x7fca1019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10106c10 0x7fca1019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10109780 0x7fca101a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca1010f5f0 con 0x7fca10109780 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca1010f470 con 0x7fca10106c10 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca18306640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca1010f770 con 0x7fca10101e80 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10109780 0x7fca101a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:34.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10109780 0x7fca101a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35426/0 (socket says 172.21.15.96:35426) 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 -- 172.21.15.96:0/2295490357 learned_addr learned my addr 172.21.15.96:0/2295490357 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca158a1640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10106c10 0x7fca1019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca160a2640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca10101e80 0x7fca1019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 -- 172.21.15.96:0/2295490357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca10101e80 msgr2=0x7fca1019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca10101e80 0x7fca1019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 -- 172.21.15.96:0/2295490357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10106c10 msgr2=0x7fca1019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10106c10 0x7fca1019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 -- 172.21.15.96:0/2295490357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca101a3f80 con 0x7fca10109780 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.569+0000 7fca168a3640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10109780 0x7fca101a3910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d950 tx=0x7fca0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.570+0000 7fca0f7fe640 1 -- 172.21.15.96:0/2295490357 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00017030 con 0x7fca10109780 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.570+0000 7fca0f7fe640 1 -- 172.21.15.96:0/2295490357 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca00004020 con 0x7fca10109780 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.570+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca101a4270 con 0x7fca10109780 2024-04-25T17:07:34.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.570+0000 7fca0f7fe640 1 -- 172.21.15.96:0/2295490357 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00003b50 con 0x7fca10109780 2024-04-25T17:07:34.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.570+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca101a47c0 con 0x7fca10109780 2024-04-25T17:07:34.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.571+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca101a4a80 con 0x7fca10109780 2024-04-25T17:07:34.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.572+0000 7fca0f7fe640 1 -- 172.21.15.96:0/2295490357 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fca0001a070 con 0x7fca10109780 2024-04-25T17:07:34.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.573+0000 7fca0f7fe640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc9f80afaf0 0x7fc9f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:34.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.573+0000 7fca160a2640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc9f80afaf0 0x7fc9f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:34.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.573+0000 7fca0f7fe640 1 -- 172.21.15.96:0/2295490357 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fca000ddfa0 con 0x7fca10109780 2024-04-25T17:07:34.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.574+0000 7fca160a2640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc9f80afaf0 0x7fc9f80b2000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca10069e60 tx=0x7fca08005c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:34.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.582+0000 7fca0f7fe640 1 -- 172.21.15.96:0/2295490357 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca00021070 con 0x7fca10109780 2024-04-25T17:07:34.626 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.171:6832/3848941544,v1:172.21.15.171:6833/3848941544] up:boot 2024-04-25T17:07:34.626 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: fsmap 1 up:standby 2024-04-25T17:07:34.626 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T17:07:34.626 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:34.627 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:34.627 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:34.627 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:34.627 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:34.627 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:34.627 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:34 smithi097 ceph-mon[32602]: Deploying daemon mds.k on smithi097 2024-04-25T17:07:34.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.798+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca101a4c50 con 0x7fca10109780 2024-04-25T17:07:34.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.799+0000 7fca0f7fe640 1 -- 172.21.15.96:0/2295490357 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fca000a7020 con 0x7fca10109780 2024-04-25T17:07:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:34.799 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T17:07:34.799 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 2 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc9f80afaf0 msgr2=0x7fc9f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc9f80afaf0 0x7fc9f80b2000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca10069e60 tx=0x7fca08005c60 comp rx=0 tx=0).stop 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10109780 msgr2=0x7fca101a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10109780 0x7fca101a3910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d950 tx=0x7fca0000de10 comp rx=0 tx=0).stop 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 shutdown_connections 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca10109780 0x7fca101a3910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca10106c10 0x7fca1019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc9f80afaf0 0x7fc9f80b2000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 --2- 172.21.15.96:0/2295490357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca10101e80 0x7fca1019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:34.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 >> 172.21.15.96:0/2295490357 conn(0x7fca100fd330 msgr2=0x7fca10103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:34.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 shutdown_connections 2024-04-25T17:07:34.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:34.801+0000 7fca18306640 1 -- 172.21.15.96:0/2295490357 wait complete. 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.171:6832/3848941544,v1:172.21.15.171:6833/3848941544] up:boot 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: fsmap 1 up:standby 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:34.811 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:34.812 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:34 smithi096 ceph-mon[29131]: Deploying daemon mds.k on smithi097 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.171:6832/3848941544,v1:172.21.15.171:6833/3848941544] up:boot 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: fsmap 1 up:standby 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:35.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:34 smithi171 ceph-mon[33189]: Deploying daemon mds.k on smithi097 2024-04-25T17:07:35.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:35 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2295490357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:35.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:35 smithi097 ceph-mon[32602]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:35.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:35 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2295490357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:35.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:35 smithi096 ceph-mon[29131]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:36.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:35 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2295490357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:36.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:35 smithi171 ceph-mon[33189]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:36.181 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:35 smithi097 systemd[1]: Starting Ceph mds.k for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:36.483 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 podman[61872]: 2024-04-25 17:07:36.085478807 +0000 UTC m=+0.018273484 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:36.483 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 podman[61872]: 2024-04-25 17:07:36.223035421 +0000 UTC m=+0.155830095 container create 900d491e8a39cc7e923e9ebdb0e509c322579f40f78eae03211edf7fff2f8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-25T17:07:36.483 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 podman[61872]: 2024-04-25 17:07:36.48436963 +0000 UTC m=+0.417164304 container init 900d491e8a39cc7e923e9ebdb0e509c322579f40f78eae03211edf7fff2f8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T17:07:36.733 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 podman[61872]: 2024-04-25 17:07:36.487055735 +0000 UTC m=+0.419850417 container start 900d491e8a39cc7e923e9ebdb0e509c322579f40f78eae03211edf7fff2f8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T17:07:36.733 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 ceph-mds[61891]: -- 172.21.15.97:0/500057852 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559af5221a40 con 0x559af5269180 2024-04-25T17:07:36.733 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k[61887]: starting mds.k at 2024-04-25T17:07:36.734 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 bash[61872]: 900d491e8a39cc7e923e9ebdb0e509c322579f40f78eae03211edf7fff2f8aff 2024-04-25T17:07:36.734 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:07:36 smithi097 systemd[1]: Started Ceph mds.k for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:36.813 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:36.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 -- 172.21.15.96:0/3943273515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e8103990 msgr2=0x7fd0e8103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 --2- 172.21.15.96:0/3943273515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e8103990 0x7fd0e8103e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009900 tx=0x7fd0d402a410 comp rx=0 tx=0).stop 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 -- 172.21.15.96:0/3943273515 shutdown_connections 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 --2- 172.21.15.96:0/3943273515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0e81044d0 0x7fd0e810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 --2- 172.21.15.96:0/3943273515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e8103990 0x7fd0e8103e40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 --2- 172.21.15.96:0/3943273515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e8102fb0 0x7fd0e81033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 -- 172.21.15.96:0/3943273515 >> 172.21.15.96:0/3943273515 conn(0x7fd0e80fd3b0 msgr2=0x7fd0e80ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 -- 172.21.15.96:0/3943273515 shutdown_connections 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 -- 172.21.15.96:0/3943273515 wait complete. 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 Processor -- start 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.925+0000 7fd0eec51640 1 -- start start 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0eec51640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0e8102fb0 0x7fd0e8102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0eec51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e8103990 0x7fd0e819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0eec51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e81044d0 0x7fd0e81a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:36.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0eec51640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810e510 con 0x7fd0e81044d0 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0eec51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810e390 con 0x7fd0e8103990 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0eec51640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810e690 con 0x7fd0e8102fb0 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e81044d0 0x7fd0e81a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e81044d0 0x7fd0e81a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35440/0 (socket says 172.21.15.96:35440) 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 -- 172.21.15.96:0/2729632662 learned_addr learned my addr 172.21.15.96:0/2729632662 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0e7fff640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e8103990 0x7fd0e819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ec9ed640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0e8102fb0 0x7fd0e8102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 -- 172.21.15.96:0/2729632662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0e8102fb0 msgr2=0x7fd0e8102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0e8102fb0 0x7fd0e8102250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 -- 172.21.15.96:0/2729632662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e8103990 msgr2=0x7fd0e819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e8103990 0x7fd0e819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 -- 172.21.15.96:0/2729632662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e81a1c60 con 0x7fd0e81044d0 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0e7fff640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e8103990 0x7fd0e819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:07:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0ed1ee640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e81044d0 0x7fd0e81a14d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000cce0 tx=0x7fd0e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0e5ffb640 1 -- 172.21.15.96:0/2729632662 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e000f450 con 0x7fd0e81044d0 2024-04-25T17:07:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0e5ffb640 1 -- 172.21.15.96:0/2729632662 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e000faa0 con 0x7fd0e81044d0 2024-04-25T17:07:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e81a1f50 con 0x7fd0e81044d0 2024-04-25T17:07:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.926+0000 7fd0e5ffb640 1 -- 172.21.15.96:0/2729632662 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0018b70 con 0x7fd0e81044d0 2024-04-25T17:07:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.927+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e81a23b0 con 0x7fd0e81044d0 2024-04-25T17:07:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.928+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e806a1c0 con 0x7fd0e81044d0 2024-04-25T17:07:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.929+0000 7fd0e5ffb640 1 -- 172.21.15.96:0/2729632662 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd0e0018d10 con 0x7fd0e81044d0 2024-04-25T17:07:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.930+0000 7fd0e5ffb640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd0d80afb70 0x7fd0d80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.930+0000 7fd0ec9ed640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd0d80afb70 0x7fd0d80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.930+0000 7fd0e5ffb640 1 -- 172.21.15.96:0/2729632662 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fd0e00dece0 con 0x7fd0e81044d0 2024-04-25T17:07:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.931+0000 7fd0ec9ed640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd0d80afb70 0x7fd0d80b2080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005d00 tx=0x7fd0dc005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:36.934+0000 7fd0e5ffb640 1 -- 172.21.15.96:0/2729632662 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0e806a1c0 con 0x7fd0e81044d0 2024-04-25T17:07:37.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.159+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0e802dc10 con 0x7fd0e81044d0 2024-04-25T17:07:37.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.160+0000 7fd0e5ffb640 1 -- 172.21.15.96:0/2729632662 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fd0e0014030 con 0x7fd0e81044d0 2024-04-25T17:07:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:37.160 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T17:07:37.160 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-04-25T17:07:37.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd0d80afb70 msgr2=0x7fd0d80b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:37.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd0d80afb70 0x7fd0d80b2080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc005d00 tx=0x7fd0dc005c50 comp rx=0 tx=0).stop 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e81044d0 msgr2=0x7fd0e81a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e81044d0 0x7fd0e81a14d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000cce0 tx=0x7fd0e000a620 comp rx=0 tx=0).stop 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 shutdown_connections 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0e81044d0 0x7fd0e81a14d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e8103990 0x7fd0e819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd0d80afb70 0x7fd0d80b2080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 --2- 172.21.15.96:0/2729632662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0e8102fb0 0x7fd0e8102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 >> 172.21.15.96:0/2729632662 conn(0x7fd0e80fd3b0 msgr2=0x7fd0e8104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 shutdown_connections 2024-04-25T17:07:37.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:37.163+0000 7fd0eec51640 1 -- 172.21.15.96:0/2729632662 wait complete. 2024-04-25T17:07:37.915 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6832/3336646389,v1:172.21.15.97:6833/3336646389] up:boot 2024-04-25T17:07:37.915 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: fsmap 2 up:standby 2024-04-25T17:07:37.915 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T17:07:37.915 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:37.915 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:37.915 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:37.916 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:37.916 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:37.916 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:37.916 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: Deploying daemon mds.j on smithi096 2024-04-25T17:07:37.916 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:37.916 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:37 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2729632662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6832/3336646389,v1:172.21.15.97:6833/3336646389] up:boot 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: fsmap 2 up:standby 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: Deploying daemon mds.j on smithi096 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:37.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:37 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2729632662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:38.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6832/3336646389,v1:172.21.15.97:6833/3336646389] up:boot 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: fsmap 2 up:standby 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: Deploying daemon mds.j on smithi096 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:38.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:37 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2729632662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:39.174 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:38 smithi096 systemd[1]: Starting Ceph mds.j for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:39.176 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:39.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.288+0000 7f8c5b89f640 1 -- 172.21.15.96:0/261110760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 msgr2=0x7f8c54104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:39.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.288+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/261110760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c54104800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009900 tx=0x7f8c5002a410 comp rx=0 tx=0).stop 2024-04-25T17:07:39.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.288+0000 7f8c5b89f640 1 -- 172.21.15.96:0/261110760 shutdown_connections 2024-04-25T17:07:39.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.288+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/261110760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c54109a40 0x7f8c5410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.288+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/261110760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c54104d40 0x7f8c54105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.288+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/261110760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c54104800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.288+0000 7f8c5b89f640 1 -- 172.21.15.96:0/261110760 >> 172.21.15.96:0/261110760 conn(0x7f8c540fed10 msgr2=0x7f8c541011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:39.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 -- 172.21.15.96:0/261110760 shutdown_connections 2024-04-25T17:07:39.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 -- 172.21.15.96:0/261110760 wait complete. 2024-04-25T17:07:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 Processor -- start 2024-04-25T17:07:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 -- start start 2024-04-25T17:07:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c5419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c54104d40 0x7f8c5419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c54109a40 0x7f8c541a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5410f600 con 0x7f8c54103b00 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5410f480 con 0x7f8c54109a40 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.289+0000 7f8c5b89f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5410f780 con 0x7f8c54104d40 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c5419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c5419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44908/0 (socket says 172.21.15.96:44908) 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 -- 172.21.15.96:0/3016694974 learned_addr learned my addr 172.21.15.96:0/3016694974 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c59e3c640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c54109a40 0x7f8c541a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c58e3a640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c54104d40 0x7f8c5419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 -- 172.21.15.96:0/3016694974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c54104d40 msgr2=0x7f8c5419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c54104d40 0x7f8c5419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 -- 172.21.15.96:0/3016694974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c54109a40 msgr2=0x7f8c541a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c54109a40 0x7f8c541a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 -- 172.21.15.96:0/3016694974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c541a1db0 con 0x7f8c54103b00 2024-04-25T17:07:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c59e3c640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c54109a40 0x7f8c541a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5963b640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c5419a2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006010 tx=0x7f8c50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c4a7fc640 1 -- 172.21.15.96:0/3016694974 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5003f030 con 0x7f8c54103b00 2024-04-25T17:07:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c4a7fc640 1 -- 172.21.15.96:0/3016694974 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c50003aa0 con 0x7f8c54103b00 2024-04-25T17:07:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c4a7fc640 1 -- 172.21.15.96:0/3016694974 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50035070 con 0x7f8c54103b00 2024-04-25T17:07:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c541a2040 con 0x7f8c54103b00 2024-04-25T17:07:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.290+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541a24e0 con 0x7f8c54103b00 2024-04-25T17:07:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.291+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c54108850 con 0x7f8c54103b00 2024-04-25T17:07:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.293+0000 7f8c4a7fc640 1 -- 172.21.15.96:0/3016694974 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8c50042030 con 0x7f8c54103b00 2024-04-25T17:07:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.294+0000 7f8c4a7fc640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8c400afb70 0x7f8c400b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.294+0000 7f8c58e3a640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8c400afb70 0x7f8c400b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.294+0000 7f8c4a7fc640 1 -- 172.21.15.96:0/3016694974 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8c500ff1d0 con 0x7f8c54103b00 2024-04-25T17:07:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.295+0000 7f8c58e3a640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8c400afb70 0x7f8c400b2080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8c44005fd0 tx=0x7f8c44005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:39.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.298+0000 7f8c4a7fc640 1 -- 172.21.15.96:0/3016694974 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c500c8250 con 0x7f8c54103b00 2024-04-25T17:07:39.438 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 podman[69420]: 2024-04-25 17:07:39.078019151 +0000 UTC m=+0.018122737 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:39.438 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 podman[69420]: 2024-04-25 17:07:39.201603865 +0000 UTC m=+0.141707426 container create f8b3ae12fee44ac8ee224fbc782eb74590486ed4cbb18d1ec9b4c188ff9258a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T17:07:39.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.523+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c5402db50 con 0x7f8c54103b00 2024-04-25T17:07:39.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.523+0000 7f8c4a7fc640 1 -- 172.21.15.96:0/3016694974 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f8c500d02d0 con 0x7f8c54103b00 2024-04-25T17:07:39.525 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:39.525 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T17:07:39.526 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-04-25T17:07:39.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8c400afb70 msgr2=0x7f8c400b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:39.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8c400afb70 0x7f8c400b2080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8c44005fd0 tx=0x7f8c44005850 comp rx=0 tx=0).stop 2024-04-25T17:07:39.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 msgr2=0x7f8c5419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:39.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c5419a2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006010 tx=0x7f8c50004270 comp rx=0 tx=0).stop 2024-04-25T17:07:39.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 shutdown_connections 2024-04-25T17:07:39.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c54109a40 0x7f8c541a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8c400afb70 0x7f8c400b2080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c54104d40 0x7f8c5419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 --2- 172.21.15.96:0/3016694974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c54103b00 0x7f8c5419a2c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:39.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 >> 172.21.15.96:0/3016694974 conn(0x7f8c540fed10 msgr2=0x7f8c54105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:39.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 shutdown_connections 2024-04-25T17:07:39.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:39.526+0000 7f8c5b89f640 1 -- 172.21.15.96:0/3016694974 wait complete. 2024-04-25T17:07:39.938 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 podman[69420]: 2024-04-25 17:07:39.490683076 +0000 UTC m=+0.430786647 container init f8b3ae12fee44ac8ee224fbc782eb74590486ed4cbb18d1ec9b4c188ff9258a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T17:07:39.938 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 podman[69420]: 2024-04-25 17:07:39.493898669 +0000 UTC m=+0.434002232 container start f8b3ae12fee44ac8ee224fbc782eb74590486ed4cbb18d1ec9b4c188ff9258a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T17:07:39.938 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 ceph-mds[69480]: -- 172.21.15.96:0/2253067061 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568b7309a40 con 0x5568b7351180 2024-04-25T17:07:39.938 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j[69474]: starting mds.j at 2024-04-25T17:07:39.938 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 bash[69420]: f8b3ae12fee44ac8ee224fbc782eb74590486ed4cbb18d1ec9b4c188ff9258a6 2024-04-25T17:07:39.938 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:07:39 smithi096 systemd[1]: Started Ceph mds.j for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3016694974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:40.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:40 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3016694974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:40.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:40.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:40 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3016694974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:40 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:41.354 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:41 smithi096 ceph-mon[29131]: Deploying daemon mds.d on smithi096 2024-04-25T17:07:41.354 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:41 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.96:6834/2619590214,v1:172.21.15.96:6835/2619590214] up:boot 2024-04-25T17:07:41.354 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:41 smithi096 ceph-mon[29131]: fsmap 3 up:standby 2024-04-25T17:07:41.354 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:41 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T17:07:41.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:41 smithi097 ceph-mon[32602]: Deploying daemon mds.d on smithi096 2024-04-25T17:07:41.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:41 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.96:6834/2619590214,v1:172.21.15.96:6835/2619590214] up:boot 2024-04-25T17:07:41.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:41 smithi097 ceph-mon[32602]: fsmap 3 up:standby 2024-04-25T17:07:41.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:41 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T17:07:41.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:41 smithi171 ceph-mon[33189]: Deploying daemon mds.d on smithi096 2024-04-25T17:07:41.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:41 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.96:6834/2619590214,v1:172.21.15.96:6835/2619590214] up:boot 2024-04-25T17:07:41.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:41 smithi171 ceph-mon[33189]: fsmap 3 up:standby 2024-04-25T17:07:41.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:41 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T17:07:41.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 -- 172.21.15.96:0/1324348335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 msgr2=0x7f81c410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 --2- 172.21.15.96:0/1324348335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c410ad40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009ca0 tx=0x7f81bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 -- 172.21.15.96:0/1324348335 shutdown_connections 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 --2- 172.21.15.96:0/1324348335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c410ad40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 --2- 172.21.15.96:0/1324348335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81c4103c20 0x7f81c4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 --2- 172.21.15.96:0/1324348335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c41029e0 0x7f81c41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 -- 172.21.15.96:0/1324348335 >> 172.21.15.96:0/1324348335 conn(0x7f81c40fdbd0 msgr2=0x7f81c4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 -- 172.21.15.96:0/1324348335 shutdown_connections 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.702+0000 7f81cb772640 1 -- 172.21.15.96:0/1324348335 wait complete. 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 Processor -- start 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 -- start start 2024-04-25T17:07:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c41029e0 0x7f81c419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81c4103c20 0x7f81c419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c41a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81c410e4f0 con 0x7f81c4108920 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81c410e370 con 0x7f81c41029e0 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81cb772640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f81c410e670 con 0x7f81c4103c20 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c41a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c41a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44956/0 (socket says 172.21.15.96:44956) 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 -- 172.21.15.96:0/165127642 learned_addr learned my addr 172.21.15.96:0/165127642 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c8d0d640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81c4103c20 0x7f81c419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c950e640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c41029e0 0x7f81c419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 -- 172.21.15.96:0/165127642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81c4103c20 msgr2=0x7f81c419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81c4103c20 0x7f81c419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 -- 172.21.15.96:0/165127642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c41029e0 msgr2=0x7f81c419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c41029e0 0x7f81c419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 -- 172.21.15.96:0/165127642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c41a1ca0 con 0x7f81c4108920 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c950e640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c41029e0 0x7f81c419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.703+0000 7f81c9d0f640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c41a1630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0097b0 tx=0x7f81bc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.704+0000 7f81ba7fc640 1 -- 172.21.15.96:0/165127642 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc004240 con 0x7f81c4108920 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.704+0000 7f81ba7fc640 1 -- 172.21.15.96:0/165127642 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81bc02f380 con 0x7f81c4108920 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.704+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c41a1f30 con 0x7f81c4108920 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.704+0000 7f81ba7fc640 1 -- 172.21.15.96:0/165127642 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc02fc30 con 0x7f81c4108920 2024-04-25T17:07:41.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.704+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41a2390 con 0x7f81c4108920 2024-04-25T17:07:41.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.705+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c41a25f0 con 0x7f81c4108920 2024-04-25T17:07:41.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.706+0000 7f81ba7fc640 1 -- 172.21.15.96:0/165127642 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f81bc0043e0 con 0x7f81c4108920 2024-04-25T17:07:41.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.707+0000 7f81ba7fc640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f81b00afaf0 0x7f81b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:41.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.707+0000 7f81c950e640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f81b00afaf0 0x7f81b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:41.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.707+0000 7f81ba7fc640 1 -- 172.21.15.96:0/165127642 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f81bc0fffa0 con 0x7f81c4108920 2024-04-25T17:07:41.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.707+0000 7f81c950e640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f81b00afaf0 0x7f81b00b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005d00 tx=0x7f81c0005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:41.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.712+0000 7f81ba7fc640 1 -- 172.21.15.96:0/165127642 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81bc0c9020 con 0x7f81c4108920 2024-04-25T17:07:41.938 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:41 smithi096 systemd[1]: Starting Ceph mds.d for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:41.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.943+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81c41a27c0 con 0x7f81c4108920 2024-04-25T17:07:41.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.944+0000 7f81ba7fc640 1 -- 172.21.15.96:0/165127642 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f81bc0098b0 con 0x7f81c4108920 2024-04-25T17:07:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:41.943 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T17:07:41.944 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 4 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f81b00afaf0 msgr2=0x7f81b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f81b00afaf0 0x7f81b00b2000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005d00 tx=0x7f81c0005ef0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 msgr2=0x7f81c41a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c41a1630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0097b0 tx=0x7f81bc003c20 comp rx=0 tx=0).stop 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 shutdown_connections 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81c4108920 0x7f81c41a1630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f81b00afaf0 0x7f81b00b2000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81c4103c20 0x7f81c419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 --2- 172.21.15.96:0/165127642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81c41029e0 0x7f81c419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 >> 172.21.15.96:0/165127642 conn(0x7f81c40fdbd0 msgr2=0x7f81c4109d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 shutdown_connections 2024-04-25T17:07:41.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:41.947+0000 7f81cb772640 1 -- 172.21.15.96:0/165127642 wait complete. 2024-04-25T17:07:42.223 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:42 smithi096 ceph-mon[29131]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:42.223 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:42 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/165127642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:42.223 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 podman[69813]: 2024-04-25 17:07:41.939571555 +0000 UTC m=+0.019141222 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:42.223 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 podman[69813]: 2024-04-25 17:07:42.083309969 +0000 UTC m=+0.162879624 container create eea5ecc06d43160d0c028de4816e1b4a734f31af572c57150c7be5b847fe0c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:07:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:42 smithi097 ceph-mon[32602]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:42 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/165127642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:42.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:42 smithi171 ceph-mon[33189]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:42.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:42 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/165127642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:42.688 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 podman[69813]: 2024-04-25 17:07:42.336251895 +0000 UTC m=+0.415821551 container init eea5ecc06d43160d0c028de4816e1b4a734f31af572c57150c7be5b847fe0c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T17:07:42.688 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 podman[69813]: 2024-04-25 17:07:42.338877669 +0000 UTC m=+0.418447326 container start eea5ecc06d43160d0c028de4816e1b4a734f31af572c57150c7be5b847fe0c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T17:07:42.688 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 ceph-mds[69834]: -- 172.21.15.96:0/4050022005 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a55eb95a40 con 0x55a55ebdd600 2024-04-25T17:07:42.688 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d[69830]: starting mds.d at 2024-04-25T17:07:42.688 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 bash[69813]: eea5ecc06d43160d0c028de4816e1b4a734f31af572c57150c7be5b847fe0c4f 2024-04-25T17:07:42.688 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:07:42 smithi096 systemd[1]: Started Ceph mds.d for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:43.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: Deploying daemon mds.e on smithi097 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.96:6836/2018971567,v1:172.21.15.96:6837/2018971567] up:boot 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: fsmap 4 up:standby 2024-04-25T17:07:43.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:43 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T17:07:43.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: Deploying daemon mds.e on smithi097 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.96:6836/2018971567,v1:172.21.15.96:6837/2018971567] up:boot 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: fsmap 4 up:standby 2024-04-25T17:07:43.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:43 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: Deploying daemon mds.e on smithi097 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:43.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.96:6836/2018971567,v1:172.21.15.96:6837/2018971567] up:boot 2024-04-25T17:07:43.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: fsmap 4 up:standby 2024-04-25T17:07:43.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:43 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T17:07:43.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:44.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.083+0000 7f60434f9640 1 -- 172.21.15.96:0/3039340873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c108920 msgr2=0x7f603c10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:44.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.083+0000 7f60434f9640 1 --2- 172.21.15.96:0/3039340873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c108920 0x7f603c10ad40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6034009ca0 tx=0x7f603402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- 172.21.15.96:0/3039340873 shutdown_connections 2024-04-25T17:07:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 --2- 172.21.15.96:0/3039340873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c108920 0x7f603c10ad40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 --2- 172.21.15.96:0/3039340873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c103c20 0x7f603c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 --2- 172.21.15.96:0/3039340873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f603c1029e0 0x7f603c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- 172.21.15.96:0/3039340873 >> 172.21.15.96:0/3039340873 conn(0x7f603c0fdbd0 msgr2=0x7f603c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- 172.21.15.96:0/3039340873 shutdown_connections 2024-04-25T17:07:44.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- 172.21.15.96:0/3039340873 wait complete. 2024-04-25T17:07:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 Processor -- start 2024-04-25T17:07:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- start start 2024-04-25T17:07:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f603c1029e0 0x7f603c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c103c20 0x7f603c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c108920 0x7f603c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10e510 con 0x7f603c103c20 2024-04-25T17:07:44.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10e390 con 0x7f603c108920 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.084+0000 7f60434f9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f603c10e690 con 0x7f603c1029e0 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c103c20 0x7f603c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c103c20 0x7f603c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44982/0 (socket says 172.21.15.96:44982) 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 -- 172.21.15.96:0/7214128 learned_addr learned my addr 172.21.15.96:0/7214128 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6041a96640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c108920 0x7f603c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6041295640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f603c1029e0 0x7f603c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 -- 172.21.15.96:0/7214128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f603c1029e0 msgr2=0x7f603c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f603c1029e0 0x7f603c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 -- 172.21.15.96:0/7214128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c108920 msgr2=0x7f603c1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c108920 0x7f603c1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 -- 172.21.15.96:0/7214128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c1a1d80 con 0x7f603c103c20 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f6040a94640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c103c20 0x7f603c19f010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f603800d950 tx=0x7f603800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f60327fc640 1 -- 172.21.15.96:0/7214128 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038017070 con 0x7f603c103c20 2024-04-25T17:07:44.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f60327fc640 1 -- 172.21.15.96:0/7214128 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6038009c30 con 0x7f603c103c20 2024-04-25T17:07:44.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c1a2070 con 0x7f603c103c20 2024-04-25T17:07:44.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f60327fc640 1 -- 172.21.15.96:0/7214128 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603800f620 con 0x7f603c103c20 2024-04-25T17:07:44.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.085+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c1a25c0 con 0x7f603c103c20 2024-04-25T17:07:44.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.086+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c107730 con 0x7f603c103c20 2024-04-25T17:07:44.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.088+0000 7f60327fc640 1 -- 172.21.15.96:0/7214128 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6038010460 con 0x7f603c103c20 2024-04-25T17:07:44.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.088+0000 7f60327fc640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f60280afaf0 0x7f60280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:44.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.089+0000 7f6041295640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f60280afaf0 0x7f60280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:44.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.089+0000 7f60327fc640 1 -- 172.21.15.96:0/7214128 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f60380de2d0 con 0x7f603c103c20 2024-04-25T17:07:44.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.089+0000 7f6041295640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f60280afaf0 0x7f60280b2000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f603c1035e0 tx=0x7f602c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:44.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.093+0000 7f60327fc640 1 -- 172.21.15.96:0/7214128 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6038009de0 con 0x7f603c103c20 2024-04-25T17:07:44.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.305+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f603c02db50 con 0x7f603c103c20 2024-04-25T17:07:44.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.306+0000 7f60327fc640 1 -- 172.21.15.96:0/7214128 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f60380a6ce0 con 0x7f603c103c20 2024-04-25T17:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T17:07:44.306 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 5 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f60280afaf0 msgr2=0x7f60280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f60280afaf0 0x7f60280b2000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f603c1035e0 tx=0x7f602c005f90 comp rx=0 tx=0).stop 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c103c20 msgr2=0x7f603c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c103c20 0x7f603c19f010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f603800d950 tx=0x7f603800de10 comp rx=0 tx=0).stop 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 shutdown_connections 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f603c108920 0x7f603c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f603c103c20 0x7f603c19f010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f60280afaf0 0x7f60280b2000 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 --2- 172.21.15.96:0/7214128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f603c1029e0 0x7f603c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 >> 172.21.15.96:0/7214128 conn(0x7f603c0fdbd0 msgr2=0x7f603c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:44.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 shutdown_connections 2024-04-25T17:07:44.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:44.308+0000 7f60434f9640 1 -- 172.21.15.96:0/7214128 wait complete. 2024-04-25T17:07:44.552 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:44 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/7214128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:44.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:44 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/7214128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:44.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:44 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/7214128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:44.948 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:44 smithi097 systemd[1]: Starting Ceph mds.e for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:45.214 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:44 smithi097 podman[62195]: 2024-04-25 17:07:44.852935622 +0000 UTC m=+0.018225612 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:45.214 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:44 smithi097 podman[62195]: 2024-04-25 17:07:44.962243507 +0000 UTC m=+0.127533479 container create 5f451bd1b1d4f1e83622efd80814d738efbd9a91213f5975981a16d0d8db0def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T17:07:45.214 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:45 smithi097 podman[62195]: 2024-04-25 17:07:45.21537832 +0000 UTC m=+0.380668290 container init 5f451bd1b1d4f1e83622efd80814d738efbd9a91213f5975981a16d0d8db0def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T17:07:45.483 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:45 smithi097 podman[62195]: 2024-04-25 17:07:45.218171302 +0000 UTC m=+0.383461271 container start 5f451bd1b1d4f1e83622efd80814d738efbd9a91213f5975981a16d0d8db0def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T17:07:45.483 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mds[62214]: -- 172.21.15.97:0/3172980083 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5566012b9a40 con 0x556601301600 2024-04-25T17:07:45.483 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e[62210]: starting mds.e at 2024-04-25T17:07:45.484 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:45 smithi097 bash[62195]: 5f451bd1b1d4f1e83622efd80814d738efbd9a91213f5975981a16d0d8db0def 2024-04-25T17:07:45.484 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:07:45 smithi097 systemd[1]: Started Ceph mds.e for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:45.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:45 smithi171 ceph-mon[33189]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:45.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:45.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:45.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:45 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:45.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:45 smithi096 ceph-mon[29131]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:45.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:45.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:45.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:45 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mon[32602]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:45.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:45.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:45.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:45 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:46.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 -- 172.21.15.96:0/886395240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c103c40 msgr2=0x7f678c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 --2- 172.21.15.96:0/886395240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c103c40 0x7f678c104050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6780009900 tx=0x7f678002a410 comp rx=0 tx=0).stop 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 -- 172.21.15.96:0/886395240 shutdown_connections 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 --2- 172.21.15.96:0/886395240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f678c108940 0x7f678c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 --2- 172.21.15.96:0/886395240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c103c40 0x7f678c104050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 --2- 172.21.15.96:0/886395240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c102a00 0x7f678c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 -- 172.21.15.96:0/886395240 >> 172.21.15.96:0/886395240 conn(0x7f678c0fdbf0 msgr2=0x7f678c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 -- 172.21.15.96:0/886395240 shutdown_connections 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 -- 172.21.15.96:0/886395240 wait complete. 2024-04-25T17:07:46.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.441+0000 7f67920f9640 1 Processor -- start 2024-04-25T17:07:46.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f67920f9640 1 -- start start 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f67920f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c102a00 0x7f678c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f67920f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c103c40 0x7f678c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f67920f9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f678c108940 0x7f678c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f67920f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10e500 con 0x7f678c102a00 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f67920f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10e380 con 0x7f678c103c40 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f67920f9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10e680 con 0x7f678c108940 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c102a00 0x7f678c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c102a00 0x7f678c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44998/0 (socket says 172.21.15.96:44998) 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 -- 172.21.15.96:0/192145279 learned_addr learned my addr 172.21.15.96:0/192145279 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 -- 172.21.15.96:0/192145279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f678c108940 msgr2=0x7f678c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f678c108940 0x7f678c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 -- 172.21.15.96:0/192145279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c103c40 msgr2=0x7f678c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678affd640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c103c40 0x7f678c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c103c40 0x7f678c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 -- 172.21.15.96:0/192145279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c1a1cf0 con 0x7f678c102a00 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f678b7fe640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c102a00 0x7f678c075840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f677c005fd0 tx=0x7f677c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.442+0000 7f6788ff9640 1 -- 172.21.15.96:0/192145279 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c017070 con 0x7f678c102a00 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.443+0000 7f6788ff9640 1 -- 172.21.15.96:0/192145279 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677c009c30 con 0x7f678c102a00 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.443+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c1a1fe0 con 0x7f678c102a00 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.443+0000 7f6788ff9640 1 -- 172.21.15.96:0/192145279 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c003d10 con 0x7f678c102a00 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.443+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c1a2440 con 0x7f678c102a00 2024-04-25T17:07:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.444+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c06a1c0 con 0x7f678c102a00 2024-04-25T17:07:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.445+0000 7f6788ff9640 1 -- 172.21.15.96:0/192145279 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f677c003eb0 con 0x7f678c102a00 2024-04-25T17:07:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.446+0000 7f6788ff9640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f67740afaf0 0x7f67740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.446+0000 7f678affd640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f67740afaf0 0x7f67740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.446+0000 7f6788ff9640 1 -- 172.21.15.96:0/192145279 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f677c0decd0 con 0x7f678c102a00 2024-04-25T17:07:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.446+0000 7f678affd640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f67740afaf0 0x7f67740b2000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f678002a930 tx=0x7f6780000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:46.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.449+0000 7f6788ff9640 1 -- 172.21.15.96:0/192145279 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f678c06a1c0 con 0x7f678c102a00 2024-04-25T17:07:46.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.668+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f678c02db50 con 0x7f678c102a00 2024-04-25T17:07:46.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.668+0000 7f6788ff9640 1 -- 172.21.15.96:0/192145279 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f677c020030 con 0x7f678c102a00 2024-04-25T17:07:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:46.668 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T17:07:46.668 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-04-25T17:07:46.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.670+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f67740afaf0 msgr2=0x7f67740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:46.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.670+0000 7f67920f9640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f67740afaf0 0x7f67740b2000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f678002a930 tx=0x7f6780000bc0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.670+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c102a00 msgr2=0x7f678c075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.670+0000 7f67920f9640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c102a00 0x7f678c075840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f677c005fd0 tx=0x7f677c00dde0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 shutdown_connections 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f67740afaf0 0x7f67740b2000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f678c108940 0x7f678c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f678c103c40 0x7f678c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 --2- 172.21.15.96:0/192145279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f678c102a00 0x7f678c075840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 >> 172.21.15.96:0/192145279 conn(0x7f678c0fdbf0 msgr2=0x7f678c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 shutdown_connections 2024-04-25T17:07:46.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:46.671+0000 7f67920f9640 1 -- 172.21.15.96:0/192145279 wait complete. 2024-04-25T17:07:46.680 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:46 smithi096 ceph-mon[29131]: Deploying daemon mds.f on smithi171 2024-04-25T17:07:46.680 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:46 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6834/3531324294,v1:172.21.15.97:6835/3531324294] up:boot 2024-04-25T17:07:46.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:46 smithi096 ceph-mon[29131]: fsmap 5 up:standby 2024-04-25T17:07:46.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:46 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T17:07:46.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:46 smithi171 ceph-mon[33189]: Deploying daemon mds.f on smithi171 2024-04-25T17:07:46.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:46 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6834/3531324294,v1:172.21.15.97:6835/3531324294] up:boot 2024-04-25T17:07:46.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:46 smithi171 ceph-mon[33189]: fsmap 5 up:standby 2024-04-25T17:07:46.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:46 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T17:07:46.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:46 smithi097 ceph-mon[32602]: Deploying daemon mds.f on smithi171 2024-04-25T17:07:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:46 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6834/3531324294,v1:172.21.15.97:6835/3531324294] up:boot 2024-04-25T17:07:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:46 smithi097 ceph-mon[32602]: fsmap 5 up:standby 2024-04-25T17:07:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:46 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T17:07:47.693 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:47 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/192145279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:47.693 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:47 smithi171 ceph-mon[33189]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:47.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:47 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/192145279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:47.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:47 smithi096 ceph-mon[29131]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:47 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/192145279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:47.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:47 smithi097 ceph-mon[32602]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:48.279 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 17:07:48 smithi171 systemd[1]: Started Ceph mds.f for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:48.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1962131178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda81029c0 msgr2=0x7fbda81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1962131178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda81029c0 0x7fbda81036c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009ca0 tx=0x7fbda002a8c0 comp rx=0 tx=0).stop 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1962131178 shutdown_connections 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1962131178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbda8108900 0x7fbda810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1962131178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8103c00 0x7fbda8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1962131178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda81029c0 0x7fbda81036c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1962131178 >> 172.21.15.96:0/1962131178 conn(0x7fbda80fdbd0 msgr2=0x7fbda8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1962131178 shutdown_connections 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.806+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1962131178 wait complete. 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 Processor -- start 2024-04-25T17:07:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 -- start start 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda81029c0 0x7fbda8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda8103c00 0x7fbda81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbda8108900 0x7fbda81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810e4c0 con 0x7fbda8103c00 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810e340 con 0x7fbda8108900 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdaeed1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810e640 con 0x7fbda81029c0 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda8103c00 0x7fbda81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda8103c00 0x7fbda81996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42494/0 (socket says 172.21.15.96:42494) 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 -- 172.21.15.96:0/1400960012 learned_addr learned my addr 172.21.15.96:0/1400960012 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdacc6d640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda81029c0 0x7fbda8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdad46e640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbda8108900 0x7fbda81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 -- 172.21.15.96:0/1400960012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda81029c0 msgr2=0x7fbda8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda81029c0 0x7fbda8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 -- 172.21.15.96:0/1400960012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbda8108900 msgr2=0x7fbda81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbda8108900 0x7fbda81a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 -- 172.21.15.96:0/1400960012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda81a0c70 con 0x7fbda8103c00 2024-04-25T17:07:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbdacc6d640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda81029c0 0x7fbda8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:07:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.807+0000 7fbda7fff640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda8103c00 0x7fbda81996c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d770 tx=0x7fbd9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.808+0000 7fbda5ffb640 1 -- 172.21.15.96:0/1400960012 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90017070 con 0x7fbda8103c00 2024-04-25T17:07:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.808+0000 7fbda5ffb640 1 -- 172.21.15.96:0/1400960012 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd900054d0 con 0x7fbda8103c00 2024-04-25T17:07:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.808+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda81a0f60 con 0x7fbda8103c00 2024-04-25T17:07:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.808+0000 7fbda5ffb640 1 -- 172.21.15.96:0/1400960012 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd90004c30 con 0x7fbda8103c00 2024-04-25T17:07:48.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.808+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a1370 con 0x7fbda8103c00 2024-04-25T17:07:48.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.809+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda8107710 con 0x7fbda8103c00 2024-04-25T17:07:48.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.810+0000 7fbda5ffb640 1 -- 172.21.15.96:0/1400960012 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbd90004e10 con 0x7fbda8103c00 2024-04-25T17:07:48.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.811+0000 7fbda5ffb640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbd940afaf0 0x7fbd940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:48.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.811+0000 7fbdacc6d640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbd940afaf0 0x7fbd940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:48.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.811+0000 7fbda5ffb640 1 -- 172.21.15.96:0/1400960012 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbd900de2f0 con 0x7fbda8103c00 2024-04-25T17:07:48.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.811+0000 7fbdacc6d640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbd940afaf0 0x7fbd940b2000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbda0000f80 tx=0x7fbda0004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:48.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:48.816+0000 7fbda5ffb640 1 -- 172.21.15.96:0/1400960012 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd900a7370 con 0x7fbda8103c00 2024-04-25T17:07:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.032+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbda802db50 con 0x7fbda8103c00 2024-04-25T17:07:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.032+0000 7fbda5ffb640 1 -- 172.21.15.96:0/1400960012 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fbd900a6c70 con 0x7fbda8103c00 2024-04-25T17:07:49.031 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:49.031 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T17:07:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 7 2024-04-25T17:07:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbd940afaf0 msgr2=0x7fbd940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbd940afaf0 0x7fbd940b2000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbda0000f80 tx=0x7fbda0004a40 comp rx=0 tx=0).stop 2024-04-25T17:07:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda8103c00 msgr2=0x7fbda81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda8103c00 0x7fbda81996c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d770 tx=0x7fbd9000dc30 comp rx=0 tx=0).stop 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 shutdown_connections 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbda8108900 0x7fbda81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbda8103c00 0x7fbda81996c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbd940afaf0 0x7fbd940b2000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 --2- 172.21.15.96:0/1400960012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda81029c0 0x7fbda8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 >> 172.21.15.96:0/1400960012 conn(0x7fbda80fdbd0 msgr2=0x7fbda81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 shutdown_connections 2024-04-25T17:07:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:49.035+0000 7fbdaeed1640 1 -- 172.21.15.96:0/1400960012 wait complete. 2024-04-25T17:07:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:49.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:49.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:49.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.171:6834/1302443885,v1:172.21.15.171:6835/1302443885] up:boot 2024-04-25T17:07:49.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: fsmap 6 up:standby 2024-04-25T17:07:49.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T17:07:49.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:49 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1400960012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:49.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:49.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:49.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:49.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.171:6834/1302443885,v1:172.21.15.171:6835/1302443885] up:boot 2024-04-25T17:07:49.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: fsmap 6 up:standby 2024-04-25T17:07:49.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T17:07:49.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:49 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1400960012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.171:6834/1302443885,v1:172.21.15.171:6835/1302443885] up:boot 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: fsmap 6 up:standby 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T17:07:49.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:49 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1400960012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:50.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:50 smithi096 ceph-mon[29131]: Deploying daemon mds.c on smithi171 2024-04-25T17:07:50.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:50 smithi096 ceph-mon[29131]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:50.472 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:50 smithi171 ceph-mon[33189]: Deploying daemon mds.c on smithi171 2024-04-25T17:07:50.472 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:50 smithi171 ceph-mon[33189]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:50.472 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 systemd[1]: Starting Ceph mds.c for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:50 smithi097 ceph-mon[32602]: Deploying daemon mds.c on smithi171 2024-04-25T17:07:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:50 smithi097 ceph-mon[32602]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:50.782 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 podman[64641]: 2024-04-25 17:07:50.376247577 +0000 UTC m=+0.018311961 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:50.782 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 podman[64641]: 2024-04-25 17:07:50.503513949 +0000 UTC m=+0.145578323 container create 7e8a63c0c96f016865c81866f29714f23e783e7e4470925063c1090836f9c9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T17:07:51.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:51.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 -- 172.21.15.96:0/826534607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 msgr2=0x7fa504104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 --2- 172.21.15.96:0/826534607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa504104010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009900 tx=0x7fa4f402a410 comp rx=0 tx=0).stop 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 -- 172.21.15.96:0/826534607 shutdown_connections 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 --2- 172.21.15.96:0/826534607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504108900 0x7fa50410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 --2- 172.21.15.96:0/826534607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa504104010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 --2- 172.21.15.96:0/826534607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5041029c0 0x7fa5041036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 -- 172.21.15.96:0/826534607 >> 172.21.15.96:0/826534607 conn(0x7fa5040fdbd0 msgr2=0x7fa504100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 -- 172.21.15.96:0/826534607 shutdown_connections 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 -- 172.21.15.96:0/826534607 wait complete. 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 Processor -- start 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.168+0000 7fa50c714640 1 -- start start 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5041029c0 0x7fa50406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa50419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:51.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504108900 0x7fa5041a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410e4c0 con 0x7fa504103c00 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410e340 con 0x7fa504108900 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa50410e640 con 0x7fa5041029c0 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa50419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa50419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42526/0 (socket says 172.21.15.96:42526) 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 -- 172.21.15.96:0/1489280626 learned_addr learned my addr 172.21.15.96:0/1489280626 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50a4b0640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5041029c0 0x7fa50406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50acb1640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504108900 0x7fa5041a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 -- 172.21.15.96:0/1489280626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5041029c0 msgr2=0x7fa50406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5041029c0 0x7fa50406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 -- 172.21.15.96:0/1489280626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504108900 msgr2=0x7fa5041a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504108900 0x7fa5041a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 -- 172.21.15.96:0/1489280626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5041a1d40 con 0x7fa504103c00 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50acb1640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504108900 0x7fa5041a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:07:51.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa509caf640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa50419efb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4006010 tx=0x7fa4f4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:51.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1489280626 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f403f030 con 0x7fa504103c00 2024-04-25T17:07:51.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1489280626 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f4003cb0 con 0x7fa504103c00 2024-04-25T17:07:51.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5041a1fd0 con 0x7fa504103c00 2024-04-25T17:07:51.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1489280626 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4035070 con 0x7fa504103c00 2024-04-25T17:07:51.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.169+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5041a2430 con 0x7fa504103c00 2024-04-25T17:07:51.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.170+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa504107710 con 0x7fa504103c00 2024-04-25T17:07:51.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.172+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1489280626 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa4f4042030 con 0x7fa504103c00 2024-04-25T17:07:51.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.173+0000 7fa4fb7fe640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4f00afaf0 0x7fa4f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:51.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.173+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1489280626 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa4f40ff510 con 0x7fa504103c00 2024-04-25T17:07:51.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.173+0000 7fa50a4b0640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4f00afaf0 0x7fa4f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:51.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.174+0000 7fa50a4b0640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4f00afaf0 0x7fa4f00b2000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5041035c0 tx=0x7fa4fc00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:51.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.177+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1489280626 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4f40c84f0 con 0x7fa504103c00 2024-04-25T17:07:51.279 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 podman[64641]: 2024-04-25 17:07:50.815361186 +0000 UTC m=+0.457425566 container init 7e8a63c0c96f016865c81866f29714f23e783e7e4470925063c1090836f9c9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:07:51.279 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 podman[64641]: 2024-04-25 17:07:50.818006027 +0000 UTC m=+0.460070415 container start 7e8a63c0c96f016865c81866f29714f23e783e7e4470925063c1090836f9c9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T17:07:51.280 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 ceph-mds[64660]: -- 172.21.15.171:0/1368833225 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3ff979a40 con 0x55a3ff9c1180 2024-04-25T17:07:51.280 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c[64656]: starting mds.c at 2024-04-25T17:07:51.280 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 bash[64641]: 7e8a63c0c96f016865c81866f29714f23e783e7e4470925063c1090836f9c9de 2024-04-25T17:07:51.280 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:07:50 smithi171 systemd[1]: Started Ceph mds.c for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:51.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.392+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa50402db50 con 0x7fa504103c00 2024-04-25T17:07:51.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.393+0000 7fa4fb7fe640 1 -- 172.21.15.96:0/1489280626 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7fa4f4003e60 con 0x7fa504103c00 2024-04-25T17:07:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:51.392 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T17:07:51.392 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 8 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4f00afaf0 msgr2=0x7fa4f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4f00afaf0 0x7fa4f00b2000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5041035c0 tx=0x7fa4fc00b390 comp rx=0 tx=0).stop 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 msgr2=0x7fa50419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa50419efb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4006010 tx=0x7fa4f4004340 comp rx=0 tx=0).stop 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 shutdown_connections 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa504108900 0x7fa5041a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa504103c00 0x7fa50419efb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4f00afaf0 0x7fa4f00b2000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 --2- 172.21.15.96:0/1489280626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5041029c0 0x7fa50406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 >> 172.21.15.96:0/1489280626 conn(0x7fa5040fdbd0 msgr2=0x7fa5041048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:51.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 shutdown_connections 2024-04-25T17:07:51.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:51.395+0000 7fa50c714640 1 -- 172.21.15.96:0/1489280626 wait complete. 2024-04-25T17:07:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: Deploying daemon mds.h on smithi097 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.171:6836/1065540086,v1:172.21.15.171:6837/1065540086] up:boot 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: fsmap 7 up:standby 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T17:07:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:51 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1489280626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:52.279 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: Deploying daemon mds.h on smithi097 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.171:6836/1065540086,v1:172.21.15.171:6837/1065540086] up:boot 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: fsmap 7 up:standby 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T17:07:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:51 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1489280626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:52.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: Deploying daemon mds.h on smithi097 2024-04-25T17:07:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.171:6836/1065540086,v1:172.21.15.171:6837/1065540086] up:boot 2024-04-25T17:07:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: fsmap 7 up:standby 2024-04-25T17:07:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T17:07:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:51 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1489280626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:53.406 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:53.432 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 systemd[1]: Starting Ceph mds.h for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:53.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 -- 172.21.15.96:0/1737359580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718109a20 msgr2=0x7f171810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 --2- 172.21.15.96:0/1737359580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718109a20 0x7f171810be40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1710009ca0 tx=0x7f171002a8c0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 -- 172.21.15.96:0/1737359580 shutdown_connections 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 --2- 172.21.15.96:0/1737359580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718109a20 0x7f171810be40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 --2- 172.21.15.96:0/1737359580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1718104d20 0x7f1718105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 --2- 172.21.15.96:0/1737359580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1718103ae0 0x7f17181047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 -- 172.21.15.96:0/1737359580 >> 172.21.15.96:0/1737359580 conn(0x7f17180fecd0 msgr2=0x7f1718101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 -- 172.21.15.96:0/1737359580 shutdown_connections 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 -- 172.21.15.96:0/1737359580 wait complete. 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.528+0000 7f171e4bd640 1 Processor -- start 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171e4bd640 1 -- start start 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171e4bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718103ae0 0x7f171819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:53.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171e4bd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1718104d20 0x7f171819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171e4bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1718109a20 0x7f17181a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171e4bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f171810f610 con 0x7f1718103ae0 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171e4bd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f171810f490 con 0x7f1718109a20 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171e4bd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f171810f790 con 0x7f1718104d20 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718103ae0 0x7f171819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718103ae0 0x7f171819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42536/0 (socket says 172.21.15.96:42536) 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 -- 172.21.15.96:0/2035195257 learned_addr learned my addr 172.21.15.96:0/2035195257 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f17177fe640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1718104d20 0x7f171819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171ca5a640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1718109a20 0x7f17181a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 -- 172.21.15.96:0/2035195257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1718104d20 msgr2=0x7f171819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1718104d20 0x7f171819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 -- 172.21.15.96:0/2035195257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1718109a20 msgr2=0x7f17181a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:53.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1718109a20 0x7f17181a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 -- 172.21.15.96:0/2035195257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17181a1c90 con 0x7f1718103ae0 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f171ca5a640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1718109a20 0x7f17181a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f17177fe640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1718104d20 0x7f171819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.529+0000 7f1717fff640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718103ae0 0x7f171819a150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f170800d740 tx=0x7f170800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.530+0000 7f17157fa640 1 -- 172.21.15.96:0/2035195257 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708017070 con 0x7f1718103ae0 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.530+0000 7f17157fa640 1 -- 172.21.15.96:0/2035195257 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f170800de50 con 0x7f1718103ae0 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.530+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17181a1f80 con 0x7f1718103ae0 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.530+0000 7f17157fa640 1 -- 172.21.15.96:0/2035195257 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708003c20 con 0x7f1718103ae0 2024-04-25T17:07:53.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.530+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17181a23e0 con 0x7f1718103ae0 2024-04-25T17:07:53.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.531+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171806afa0 con 0x7f1718103ae0 2024-04-25T17:07:53.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.532+0000 7f17157fa640 1 -- 172.21.15.96:0/2035195257 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1708003dc0 con 0x7f1718103ae0 2024-04-25T17:07:53.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.533+0000 7f17157fa640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17040afaf0 0x7f17040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:53.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.533+0000 7f17177fe640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17040afaf0 0x7f17040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:53.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.533+0000 7f17157fa640 1 -- 172.21.15.96:0/2035195257 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f17080de240 con 0x7f1718103ae0 2024-04-25T17:07:53.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.533+0000 7f17177fe640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17040afaf0 0x7f17040b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f171819b1f0 tx=0x7f1700005e70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:53.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.537+0000 7f17157fa640 1 -- 172.21.15.96:0/2035195257 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17080a72c0 con 0x7f1718103ae0 2024-04-25T17:07:53.711 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 podman[62505]: 2024-04-25 17:07:53.336558722 +0000 UTC m=+0.018383318 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:53.711 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 podman[62505]: 2024-04-25 17:07:53.46010498 +0000 UTC m=+0.141929549 container create 8ee9a110c54048c4924153b707e1f920be8be1d841cf402d0da9a2bd0778710d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T17:07:53.711 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 podman[62505]: 2024-04-25 17:07:53.712819209 +0000 UTC m=+0.394643813 container init 8ee9a110c54048c4924153b707e1f920be8be1d841cf402d0da9a2bd0778710d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T17:07:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.753+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f171802dc10 con 0x7f1718103ae0 2024-04-25T17:07:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.753+0000 7f17157fa640 1 -- 172.21.15.96:0/2035195257 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f17080a6bc0 con 0x7f1718103ae0 2024-04-25T17:07:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:53.752 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T17:07:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 8 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17040afaf0 msgr2=0x7f17040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17040afaf0 0x7f17040b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f171819b1f0 tx=0x7f1700005e70 comp rx=0 tx=0).stop 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718103ae0 msgr2=0x7f171819a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718103ae0 0x7f171819a150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f170800d740 tx=0x7f170800dc00 comp rx=0 tx=0).stop 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 shutdown_connections 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1718109a20 0x7f17181a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17040afaf0 0x7f17040b2000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1718104d20 0x7f171819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 --2- 172.21.15.96:0/2035195257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1718103ae0 0x7f171819a150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 >> 172.21.15.96:0/2035195257 conn(0x7f17180fecd0 msgr2=0x7f1718105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:53.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 shutdown_connections 2024-04-25T17:07:53.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:53.755+0000 7f171e4bd640 1 -- 172.21.15.96:0/2035195257 wait complete. 2024-04-25T17:07:53.983 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 podman[62505]: 2024-04-25 17:07:53.715443416 +0000 UTC m=+0.397267991 container start 8ee9a110c54048c4924153b707e1f920be8be1d841cf402d0da9a2bd0778710d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T17:07:53.983 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 ceph-mds[62524]: -- 172.21.15.97:0/406927359 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fcb073ba40 con 0x55fcb0783a80 2024-04-25T17:07:53.983 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[62520]: starting mds.h at 2024-04-25T17:07:53.983 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 bash[62505]: 8ee9a110c54048c4924153b707e1f920be8be1d841cf402d0da9a2bd0778710d 2024-04-25T17:07:53.983 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:07:53 smithi097 systemd[1]: Started Ceph mds.h for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2035195257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:54 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2035195257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:54 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2035195257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:54.522 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:54 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:55.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:55 smithi096 ceph-mon[29131]: Deploying daemon mds.i on smithi171 2024-04-25T17:07:55.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:55 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6836/825277463,v1:172.21.15.97:6837/825277463] up:boot 2024-04-25T17:07:55.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:55 smithi096 ceph-mon[29131]: fsmap 8 up:standby 2024-04-25T17:07:55.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:55 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:07:55.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:55 smithi097 ceph-mon[32602]: Deploying daemon mds.i on smithi171 2024-04-25T17:07:55.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:55 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6836/825277463,v1:172.21.15.97:6837/825277463] up:boot 2024-04-25T17:07:55.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:55 smithi097 ceph-mon[32602]: fsmap 8 up:standby 2024-04-25T17:07:55.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:55 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:07:55.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:55 smithi171 ceph-mon[33189]: Deploying daemon mds.i on smithi171 2024-04-25T17:07:55.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:55 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6836/825277463,v1:172.21.15.97:6837/825277463] up:boot 2024-04-25T17:07:55.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:55 smithi171 ceph-mon[33189]: fsmap 8 up:standby 2024-04-25T17:07:55.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:55 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:07:55.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.887+0000 7f3218abd640 1 -- 172.21.15.96:0/1291456345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 msgr2=0x7f3214105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:55.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.887+0000 7f3218abd640 1 --2- 172.21.15.96:0/1291456345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f3214105150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009900 tx=0x7f31fc02a410 comp rx=0 tx=0).stop 2024-04-25T17:07:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 -- 172.21.15.96:0/1291456345 shutdown_connections 2024-04-25T17:07:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 --2- 172.21.15.96:0/1291456345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3214109a40 0x7f321410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 --2- 172.21.15.96:0/1291456345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f3214105150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 --2- 172.21.15.96:0/1291456345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214103b00 0x7f3214104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 -- 172.21.15.96:0/1291456345 >> 172.21.15.96:0/1291456345 conn(0x7f32140fed10 msgr2=0x7f32141011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 -- 172.21.15.96:0/1291456345 shutdown_connections 2024-04-25T17:07:55.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 -- 172.21.15.96:0/1291456345 wait complete. 2024-04-25T17:07:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 Processor -- start 2024-04-25T17:07:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.888+0000 7f3218abd640 1 -- start start 2024-04-25T17:07:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3214103b00 0x7f321419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f321419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214109a40 0x7f32141a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f321410f680 con 0x7f3214104d40 2024-04-25T17:07:55.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f321410f500 con 0x7f3214103b00 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f321410f800 con 0x7f3214109a40 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f321419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f321419a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42558/0 (socket says 172.21.15.96:42558) 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 -- 172.21.15.96:0/854116576 learned_addr learned my addr 172.21.15.96:0/854116576 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f321259c640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3214103b00 0x7f321419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3212d9d640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214109a40 0x7f32141a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 -- 172.21.15.96:0/854116576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214109a40 msgr2=0x7f32141a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214109a40 0x7f32141a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 -- 172.21.15.96:0/854116576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3214103b00 msgr2=0x7f321419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3214103b00 0x7f321419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 -- 172.21.15.96:0/854116576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32141a1e00 con 0x7f3214104d40 2024-04-25T17:07:55.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f321259c640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3214103b00 0x7f321419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:07:55.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320bfff640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f321419a800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31fc006010 tx=0x7f31fc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:55.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320b7fe640 1 -- 172.21.15.96:0/854116576 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc03f030 con 0x7f3214104d40 2024-04-25T17:07:55.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320b7fe640 1 -- 172.21.15.96:0/854116576 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc003cb0 con 0x7f3214104d40 2024-04-25T17:07:55.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32141a2090 con 0x7f3214104d40 2024-04-25T17:07:55.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f320b7fe640 1 -- 172.21.15.96:0/854116576 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc035070 con 0x7f3214104d40 2024-04-25T17:07:55.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.889+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32141a2600 con 0x7f3214104d40 2024-04-25T17:07:55.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.891+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3214108850 con 0x7f3214104d40 2024-04-25T17:07:55.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.892+0000 7f320b7fe640 1 -- 172.21.15.96:0/854116576 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f31fc042030 con 0x7f3214104d40 2024-04-25T17:07:55.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.892+0000 7f320b7fe640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f32000afaf0 0x7f32000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:55.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.893+0000 7f321259c640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f32000afaf0 0x7f32000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:55.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.893+0000 7f320b7fe640 1 -- 172.21.15.96:0/854116576 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f31fc0ff0b0 con 0x7f3214104d40 2024-04-25T17:07:55.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.893+0000 7f321259c640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f32000afaf0 0x7f32000b2000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3214104700 tx=0x7f3204008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:55.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:55.896+0000 7f320b7fe640 1 -- 172.21.15.96:0/854116576 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31fc0c8130 con 0x7f3214104d40 2024-04-25T17:07:56.108 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 systemd[1]: Starting Ceph mds.i for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:56.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.111+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f321402db50 con 0x7f3214104d40 2024-04-25T17:07:56.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.112+0000 7f320b7fe640 1 -- 172.21.15.96:0/854116576 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f31fc0c7a30 con 0x7f3214104d40 2024-04-25T17:07:56.111 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:56.111 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T17:07:56.111 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-04-25T17:07:56.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f32000afaf0 msgr2=0x7f32000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:56.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f32000afaf0 0x7f32000b2000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3214104700 tx=0x7f3204008040 comp rx=0 tx=0).stop 2024-04-25T17:07:56.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 msgr2=0x7f321419a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:56.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f321419a800 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31fc006010 tx=0x7f31fc004340 comp rx=0 tx=0).stop 2024-04-25T17:07:56.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 shutdown_connections 2024-04-25T17:07:56.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f32000afaf0 0x7f32000b2000 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:56.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214109a40 0x7f32141a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:56.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3214104d40 0x7f321419a800 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:56.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 --2- 172.21.15.96:0/854116576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3214103b00 0x7f321419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:56.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 >> 172.21.15.96:0/854116576 conn(0x7f32140fed10 msgr2=0x7f3214105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:56.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 shutdown_connections 2024-04-25T17:07:56.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:56.114+0000 7f3218abd640 1 -- 172.21.15.96:0/854116576 wait complete. 2024-04-25T17:07:56.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:56 smithi171 ceph-mon[33189]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:56.398 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 podman[64949]: 2024-04-25 17:07:56.149832014 +0000 UTC m=+0.017934630 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:56.398 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 podman[64949]: 2024-04-25 17:07:56.276709926 +0000 UTC m=+0.144812516 container create 4372dd765e2bf792e57bfa7089efcd80bbeb8c33f1c1b5a792179926758712c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T17:07:56.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:56 smithi096 ceph-mon[29131]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:56 smithi097 ceph-mon[32602]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:56.779 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 podman[64949]: 2024-04-25 17:07:56.512718786 +0000 UTC m=+0.380821376 container init 4372dd765e2bf792e57bfa7089efcd80bbeb8c33f1c1b5a792179926758712c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T17:07:56.779 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 podman[64949]: 2024-04-25 17:07:56.51513883 +0000 UTC m=+0.383241425 container start 4372dd765e2bf792e57bfa7089efcd80bbeb8c33f1c1b5a792179926758712c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T17:07:56.780 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 ceph-mds[64967]: -- 172.21.15.171:0/3364168762 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55608aa95a40 con 0x55608aadd600 2024-04-25T17:07:56.780 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i[64963]: starting mds.i at 2024-04-25T17:07:56.780 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 bash[64949]: 4372dd765e2bf792e57bfa7089efcd80bbeb8c33f1c1b5a792179926758712c7 2024-04-25T17:07:56.780 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:07:56 smithi171 systemd[1]: Started Ceph mds.i for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:07:57.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:57 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/854116576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:57.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:57.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:57.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:57 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:57.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:57 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/854116576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:57.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:57.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:57.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:57 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:57.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:57 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/854116576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:57.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:07:57.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:07:57.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:07:57.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:57 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:07:58.125 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:07:58.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.236+0000 7f1ece372640 1 -- 172.21.15.96:0/2916709221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec81041f0 msgr2=0x7f1ec8106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.236+0000 7f1ece372640 1 --2- 172.21.15.96:0/2916709221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec81041f0 0x7f1ec8106610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009900 tx=0x7f1eb802a410 comp rx=0 tx=0).stop 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 -- 172.21.15.96:0/2916709221 shutdown_connections 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 --2- 172.21.15.96:0/2916709221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ec8109750 0x7f1ec810bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 --2- 172.21.15.96:0/2916709221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8106be0 0x7f1ec81090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 --2- 172.21.15.96:0/2916709221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec81041f0 0x7f1ec8106610 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 -- 172.21.15.96:0/2916709221 >> 172.21.15.96:0/2916709221 conn(0x7f1ec80fdbd0 msgr2=0x7f1ec8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 -- 172.21.15.96:0/2916709221 shutdown_connections 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 -- 172.21.15.96:0/2916709221 wait complete. 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 Processor -- start 2024-04-25T17:07:58.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.237+0000 7f1ece372640 1 -- start start 2024-04-25T17:07:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ec81041f0 0x7f1ec8103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec8106be0 0x7f1ec8102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8109750 0x7f1ec8102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810f5b0 con 0x7f1ec8106be0 2024-04-25T17:07:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810f430 con 0x7f1ec8109750 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec810f730 con 0x7f1ec81041f0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec8106be0 0x7f1ec8102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec8106be0 0x7f1ec8102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37532/0 (socket says 172.21.15.96:37532) 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 -- 172.21.15.96:0/2371798769 learned_addr learned my addr 172.21.15.96:0/2371798769 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec7fff640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ec81041f0 0x7f1ec8103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ecc90f640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8109750 0x7f1ec8102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 -- 172.21.15.96:0/2371798769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ec81041f0 msgr2=0x7f1ec8103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ec81041f0 0x7f1ec8103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 -- 172.21.15.96:0/2371798769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8109750 msgr2=0x7f1ec8102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8109750 0x7f1ec8102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 -- 172.21.15.96:0/2371798769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec8102d70 con 0x7f1ec8106be0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ecc90f640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8109750 0x7f1ec8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec77fe640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec8106be0 0x7f1ec8102100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00d950 tx=0x7f1ebc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec57fa640 1 -- 172.21.15.96:0/2371798769 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc017070 con 0x7f1ec8106be0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec57fa640 1 -- 172.21.15.96:0/2371798769 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc009c30 con 0x7f1ec8106be0 2024-04-25T17:07:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a42b0 con 0x7f1ec8106be0 2024-04-25T17:07:58.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ec57fa640 1 -- 172.21.15.96:0/2371798769 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00f620 con 0x7f1ec8106be0 2024-04-25T17:07:58.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.238+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a47a0 con 0x7f1ec8106be0 2024-04-25T17:07:58.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.239+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec80ff6f0 con 0x7f1ec8106be0 2024-04-25T17:07:58.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.242+0000 7f1ec57fa640 1 -- 172.21.15.96:0/2371798769 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1ebc010460 con 0x7f1ec8106be0 2024-04-25T17:07:58.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.242+0000 7f1ec57fa640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1eb00afb70 0x7f1eb00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:07:58.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.242+0000 7f1ec7fff640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1eb00afb70 0x7f1eb00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:07:58.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.242+0000 7f1ec57fa640 1 -- 172.21.15.96:0/2371798769 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f1ebc0de2e0 con 0x7f1ec8106be0 2024-04-25T17:07:58.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.243+0000 7f1ec7fff640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1eb00afb70 0x7f1eb00b2080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009230 tx=0x7f1eb8031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:07:58.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.246+0000 7f1ec57fa640 1 -- 172.21.15.96:0/2371798769 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ebc0a73e0 con 0x7f1ec8106be0 2024-04-25T17:07:58.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:58 smithi096 ceph-mon[29131]: Deploying daemon mds.g on smithi096 2024-04-25T17:07:58.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:58 smithi096 ceph-mon[29131]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:58.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:58 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.171:6838/2758729388,v1:172.21.15.171:6839/2758729388] up:boot 2024-04-25T17:07:58.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:58 smithi096 ceph-mon[29131]: fsmap 9 up:standby 2024-04-25T17:07:58.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:58 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T17:07:58.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.466+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ec802db50 con 0x7f1ec8106be0 2024-04-25T17:07:58.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.466+0000 7f1ec57fa640 1 -- 172.21.15.96:0/2371798769 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f1ebc009de0 con 0x7f1ec8106be0 2024-04-25T17:07:58.466 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:07:58.466 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T17:07:58.467 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 10 2024-04-25T17:07:58.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1eb00afb70 msgr2=0x7f1eb00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:58.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1eb00afb70 0x7f1eb00b2080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009230 tx=0x7f1eb8031400 comp rx=0 tx=0).stop 2024-04-25T17:07:58.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec8106be0 msgr2=0x7f1ec8102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:07:58.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec8106be0 0x7f1ec8102100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00d950 tx=0x7f1ebc00de10 comp rx=0 tx=0).stop 2024-04-25T17:07:58.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 shutdown_connections 2024-04-25T17:07:58.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec8109750 0x7f1ec8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ec8106be0 0x7f1ec8102100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1eb00afb70 0x7f1eb00b2080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 --2- 172.21.15.96:0/2371798769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ec81041f0 0x7f1ec8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:07:58.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 >> 172.21.15.96:0/2371798769 conn(0x7f1ec80fdbd0 msgr2=0x7f1ec8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:07:58.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 shutdown_connections 2024-04-25T17:07:58.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:07:58.468+0000 7f1ece372640 1 -- 172.21.15.96:0/2371798769 wait complete. 2024-04-25T17:07:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:58 smithi097 ceph-mon[32602]: Deploying daemon mds.g on smithi096 2024-04-25T17:07:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:58 smithi097 ceph-mon[32602]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:58 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.171:6838/2758729388,v1:172.21.15.171:6839/2758729388] up:boot 2024-04-25T17:07:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:58 smithi097 ceph-mon[32602]: fsmap 9 up:standby 2024-04-25T17:07:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:58 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T17:07:58.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:58 smithi171 ceph-mon[33189]: Deploying daemon mds.g on smithi096 2024-04-25T17:07:58.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:58 smithi171 ceph-mon[33189]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:07:58.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:58 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.171:6838/2758729388,v1:172.21.15.171:6839/2758729388] up:boot 2024-04-25T17:07:58.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:58 smithi171 ceph-mon[33189]: fsmap 9 up:standby 2024-04-25T17:07:58.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:58 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T17:07:58.835 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:58 smithi096 systemd[1]: Starting Ceph mds.g for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:07:59.088 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 podman[70412]: 2024-04-25 17:07:58.957306269 +0000 UTC m=+0.018011430 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:07:59.088 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 podman[70412]: 2024-04-25 17:07:59.089245901 +0000 UTC m=+0.149951089 container create 4ab184210c24cf63c80cda7a4c48174e8c8dd5325052cdddbaee0c315485f960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T17:07:59.357 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:07:59 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2371798769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:59.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:07:59 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2371798769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:59.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:07:59 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2371798769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:07:59.642 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 podman[70412]: 2024-04-25 17:07:59.358853202 +0000 UTC m=+0.419558348 container init 4ab184210c24cf63c80cda7a4c48174e8c8dd5325052cdddbaee0c315485f960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T17:07:59.642 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 podman[70412]: 2024-04-25 17:07:59.3620787 +0000 UTC m=+0.422783846 container start 4ab184210c24cf63c80cda7a4c48174e8c8dd5325052cdddbaee0c315485f960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T17:07:59.642 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 ceph-mds[70432]: -- 172.21.15.96:0/1952373357 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556dfa61da40 con 0x556dfa665600 2024-04-25T17:07:59.642 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g[70427]: starting mds.g at 2024-04-25T17:07:59.642 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 bash[70412]: 4ab184210c24cf63c80cda7a4c48174e8c8dd5325052cdddbaee0c315485f960 2024-04-25T17:07:59.642 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:07:59 smithi096 systemd[1]: Started Ceph mds.g for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:08:00.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:00 smithi096 ceph-mon[29131]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:00.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:08:00.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:08:00.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:00 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:00.480 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:08:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:00 smithi097 ceph-mon[32602]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:08:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:08:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:00 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:00 smithi171 ceph-mon[33189]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:00.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:08:00.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:08:00.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:00 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:00.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 -- 172.21.15.96:0/1243532303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 msgr2=0x7f701c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 --2- 172.21.15.96:0/1243532303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c1036e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f700c009900 tx=0x7f700c02a410 comp rx=0 tx=0).stop 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 -- 172.21.15.96:0/1243532303 shutdown_connections 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 --2- 172.21.15.96:0/1243532303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f701c108920 0x7f701c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 --2- 172.21.15.96:0/1243532303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c103c20 0x7f701c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 --2- 172.21.15.96:0/1243532303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c1036e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 -- 172.21.15.96:0/1243532303 >> 172.21.15.96:0/1243532303 conn(0x7f701c0fdbd0 msgr2=0x7f701c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 -- 172.21.15.96:0/1243532303 shutdown_connections 2024-04-25T17:08:00.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 -- 172.21.15.96:0/1243532303 wait complete. 2024-04-25T17:08:00.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 Processor -- start 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.602+0000 7f70210e9640 1 -- start start 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c103c20 0x7f701c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f701c108920 0x7f701c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10e4e0 con 0x7f701c1029e0 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10e360 con 0x7f701c103c20 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10e660 con 0x7f701c108920 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37554/0 (socket says 172.21.15.96:37554) 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 -- 172.21.15.96:0/2055219251 learned_addr learned my addr 172.21.15.96:0/2055219251 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701b59e640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f701c108920 0x7f701c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f7013fff640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c103c20 0x7f701c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:00.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 -- 172.21.15.96:0/2055219251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f701c108920 msgr2=0x7f701c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f701c108920 0x7f701c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 -- 172.21.15.96:0/2055219251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c103c20 msgr2=0x7f701c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c103c20 0x7f701c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 -- 172.21.15.96:0/2055219251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c1a1da0 con 0x7f701c1029e0 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701b59e640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f701c108920 0x7f701c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f701ad9d640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c075860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f700c006010 tx=0x7f700c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f7018d99640 1 -- 172.21.15.96:0/2055219251 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c03f030 con 0x7f701c1029e0 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c1a2030 con 0x7f701c1029e0 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f7018d99640 1 -- 172.21.15.96:0/2055219251 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f700c003aa0 con 0x7f701c1029e0 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f7018d99640 1 -- 172.21.15.96:0/2055219251 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c035070 con 0x7f701c1029e0 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.603+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a2520 con 0x7f701c1029e0 2024-04-25T17:08:00.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.604+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c107730 con 0x7f701c1029e0 2024-04-25T17:08:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.606+0000 7f7018d99640 1 -- 172.21.15.96:0/2055219251 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f700c042030 con 0x7f701c1029e0 2024-04-25T17:08:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.607+0000 7f7018d99640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70080afaf0 0x7f70080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.607+0000 7f7013fff640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70080afaf0 0x7f70080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.607+0000 7f7018d99640 1 -- 172.21.15.96:0/2055219251 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f700c0ff280 con 0x7f701c1029e0 2024-04-25T17:08:00.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.607+0000 7f7013fff640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70080afaf0 0x7f70080b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7004005d00 tx=0x7f7004005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:00.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.611+0000 7f7018d99640 1 -- 172.21.15.96:0/2055219251 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f700c0c8300 con 0x7f701c1029e0 2024-04-25T17:08:00.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.827+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f701c02dc70 con 0x7f701c1029e0 2024-04-25T17:08:00.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.827+0000 7f7018d99640 1 -- 172.21.15.96:0/2055219251 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f700c0d0380 con 0x7f701c1029e0 2024-04-25T17:08:00.827 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:00.827 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T17:08:00.828 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 11 2024-04-25T17:08:00.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70080afaf0 msgr2=0x7f70080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70080afaf0 0x7f70080b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7004005d00 tx=0x7f7004005c70 comp rx=0 tx=0).stop 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 msgr2=0x7f701c075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c075860 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f700c006010 tx=0x7f700c004270 comp rx=0 tx=0).stop 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 shutdown_connections 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70080afaf0 0x7f70080b2000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f701c108920 0x7f701c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701c103c20 0x7f701c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 --2- 172.21.15.96:0/2055219251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f701c1029e0 0x7f701c075860 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 >> 172.21.15.96:0/2055219251 conn(0x7f701c0fdbd0 msgr2=0x7f701c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 shutdown_connections 2024-04-25T17:08:00.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:00.830+0000 7f70210e9640 1 -- 172.21.15.96:0/2055219251 wait complete. 2024-04-25T17:08:01.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:01 smithi096 ceph-mon[29131]: Deploying daemon mds.a on smithi096 2024-04-25T17:08:01.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:01 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.96:6838/416971885,v1:172.21.15.96:6839/416971885] up:boot 2024-04-25T17:08:01.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:01 smithi096 ceph-mon[29131]: fsmap 10 up:standby 2024-04-25T17:08:01.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:01 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T17:08:01.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:01 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2055219251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:01 smithi097 ceph-mon[32602]: Deploying daemon mds.a on smithi096 2024-04-25T17:08:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:01 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.96:6838/416971885,v1:172.21.15.96:6839/416971885] up:boot 2024-04-25T17:08:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:01 smithi097 ceph-mon[32602]: fsmap 10 up:standby 2024-04-25T17:08:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:01 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T17:08:01.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:01 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2055219251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:01.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:01 smithi171 ceph-mon[33189]: Deploying daemon mds.a on smithi096 2024-04-25T17:08:01.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:01 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.96:6838/416971885,v1:172.21.15.96:6839/416971885] up:boot 2024-04-25T17:08:01.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:01 smithi171 ceph-mon[33189]: fsmap 10 up:standby 2024-04-25T17:08:01.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:01 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T17:08:01.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:01 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2055219251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:01.895 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:01 smithi096 systemd[1]: Starting Ceph mds.a for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:08:02.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:02 smithi096 ceph-mon[29131]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:02.148 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:01 smithi096 podman[70778]: 2024-04-25 17:08:01.799105421 +0000 UTC m=+0.018098616 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:08:02.148 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:01 smithi096 podman[70778]: 2024-04-25 17:08:01.905088717 +0000 UTC m=+0.124081898 container create 6e4ee577a3d3a806766c849dd857e0c4a62978be9b7c20ef325fd5c9f5cdc7a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T17:08:02.148 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:02 smithi096 podman[70778]: 2024-04-25 17:08:02.149843646 +0000 UTC m=+0.368836817 container init 6e4ee577a3d3a806766c849dd857e0c4a62978be9b7c20ef325fd5c9f5cdc7a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T17:08:02.384 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:02 smithi097 ceph-mon[32602]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:02.431 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:02 smithi171 ceph-mon[33189]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:02.438 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:02 smithi096 podman[70778]: 2024-04-25 17:08:02.153072669 +0000 UTC m=+0.372065848 container start 6e4ee577a3d3a806766c849dd857e0c4a62978be9b7c20ef325fd5c9f5cdc7a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:08:02.438 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:02 smithi096 ceph-mds[70797]: -- 172.21.15.96:0/3439817892 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9d4039a40 con 0x55a9d4081180 2024-04-25T17:08:02.438 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a[70793]: starting mds.a at 2024-04-25T17:08:02.438 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:02 smithi096 bash[70778]: 6e4ee577a3d3a806766c849dd857e0c4a62978be9b7c20ef325fd5c9f5cdc7a5 2024-04-25T17:08:02.438 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:08:02 smithi096 systemd[1]: Started Ceph mds.a for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:08:02.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:08:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 -- 172.21.15.96:0/4032501459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08109a50 msgr2=0x7fbd0810be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/4032501459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08109a50 0x7fbd0810be70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009ca0 tx=0x7fbd0002a8e0 comp rx=0 tx=0).stop 2024-04-25T17:08:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 -- 172.21.15.96:0/4032501459 shutdown_connections 2024-04-25T17:08:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/4032501459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08109a50 0x7fbd0810be70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:02.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/4032501459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd08104d50 0x7fbd08105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/4032501459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd08103b10 0x7fbd08104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 -- 172.21.15.96:0/4032501459 >> 172.21.15.96:0/4032501459 conn(0x7fbd080fed40 msgr2=0x7fbd081011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 -- 172.21.15.96:0/4032501459 shutdown_connections 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.957+0000 7fbd0f77a640 1 -- 172.21.15.96:0/4032501459 wait complete. 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 Processor -- start 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 -- start start 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08103b10 0x7fbd0819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd08104d50 0x7fbd0819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd08109a50 0x7fbd081a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810f690 con 0x7fbd08103b10 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810f510 con 0x7fbd08109a50 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0f77a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0810f810 con 0x7fbd08104d50 2024-04-25T17:08:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0d516640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08103b10 0x7fbd0819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0d516640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08103b10 0x7fbd0819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37582/0 (socket says 172.21.15.96:37582) 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0d516640 1 -- 172.21.15.96:0/2340652848 learned_addr learned my addr 172.21.15.96:0/2340652848 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0cd15640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd08104d50 0x7fbd0819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.958+0000 7fbd0dd17640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd08109a50 0x7fbd081a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0d516640 1 -- 172.21.15.96:0/2340652848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd08104d50 msgr2=0x7fbd0819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0d516640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd08104d50 0x7fbd0819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0d516640 1 -- 172.21.15.96:0/2340652848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd08109a50 msgr2=0x7fbd081a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0d516640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd08109a50 0x7fbd081a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0d516640 1 -- 172.21.15.96:0/2340652848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd081a1d70 con 0x7fbd08103b10 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0d516640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08103b10 0x7fbd0819a230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006fd0 tx=0x7fbd0400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbcfa7fc640 1 -- 172.21.15.96:0/2340652848 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04011830 con 0x7fbd08103b10 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbcfa7fc640 1 -- 172.21.15.96:0/2340652848 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd04016070 con 0x7fbd08103b10 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd081a2060 con 0x7fbd08103b10 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbcfa7fc640 1 -- 172.21.15.96:0/2340652848 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04010e00 con 0x7fbd08103b10 2024-04-25T17:08:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.959+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd081a2630 con 0x7fbd08103b10 2024-04-25T17:08:02.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.960+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd08108860 con 0x7fbd08103b10 2024-04-25T17:08:02.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.961+0000 7fbcfa7fc640 1 -- 172.21.15.96:0/2340652848 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fbd0401b460 con 0x7fbd08103b10 2024-04-25T17:08:02.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.962+0000 7fbcfa7fc640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcf40afaf0 0x7fbcf40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:02.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.962+0000 7fbd0cd15640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcf40afaf0 0x7fbcf40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:02.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.962+0000 7fbcfa7fc640 1 -- 172.21.15.96:0/2340652848 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbd040de3d0 con 0x7fbd08103b10 2024-04-25T17:08:02.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.963+0000 7fbd0cd15640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcf40afaf0 0x7fbcf40b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd0819b2d0 tx=0x7fbcfc00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:02.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:02.966+0000 7fbcfa7fc640 1 -- 172.21.15.96:0/2340652848 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd040a7450 con 0x7fbd08103b10 2024-04-25T17:08:03.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.185+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd0802db50 con 0x7fbd08103b10 2024-04-25T17:08:03.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.186+0000 7fbcfa7fc640 1 -- 172.21.15.96:0/2340652848 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7fbd040a6d50 con 0x7fbd08103b10 2024-04-25T17:08:03.186 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:03.186 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T17:08:03.186 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcf40afaf0 msgr2=0x7fbcf40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcf40afaf0 0x7fbcf40b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd0819b2d0 tx=0x7fbcfc00b390 comp rx=0 tx=0).stop 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08103b10 msgr2=0x7fbd0819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08103b10 0x7fbd0819a230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd04006fd0 tx=0x7fbd0400b8e0 comp rx=0 tx=0).stop 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 shutdown_connections 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd08109a50 0x7fbd081a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbcf40afaf0 0x7fbcf40b2000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd08104d50 0x7fbd0819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 --2- 172.21.15.96:0/2340652848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbd08103b10 0x7fbd0819a230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 >> 172.21.15.96:0/2340652848 conn(0x7fbd080fed40 msgr2=0x7fbd08105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 shutdown_connections 2024-04-25T17:08:03.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:03.188+0000 7fbd0f77a640 1 -- 172.21.15.96:0/2340652848 wait complete. 2024-04-25T17:08:03.631 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: Deploying daemon mds.b on smithi097 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.96:6840/1046285931,v1:172.21.15.96:6841/1046285931] up:boot 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: fsmap 11 up:standby 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T17:08:03.632 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:03 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2340652848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:03.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:08:03.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:08:03.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:03.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: Deploying daemon mds.b on smithi097 2024-04-25T17:08:03.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:03.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.96:6840/1046285931,v1:172.21.15.96:6841/1046285931] up:boot 2024-04-25T17:08:03.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: fsmap 11 up:standby 2024-04-25T17:08:03.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T17:08:03.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:03 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2340652848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:03.702 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.702 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.702 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:03.702 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T17:08:03.702 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T17:08:03.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:03.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: Deploying daemon mds.b on smithi097 2024-04-25T17:08:03.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:03.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.96:6840/1046285931,v1:172.21.15.96:6841/1046285931] up:boot 2024-04-25T17:08:03.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: fsmap 11 up:standby 2024-04-25T17:08:03.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T17:08:03.703 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:03 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2340652848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:04.734 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:04 smithi097 systemd[1]: Starting Ceph mds.b for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:08:05.058 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:04 smithi097 podman[62821]: 2024-04-25 17:08:04.682722763 +0000 UTC m=+0.018101784 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:08:05.058 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:04 smithi097 podman[62821]: 2024-04-25 17:08:04.798845916 +0000 UTC m=+0.134224939 container create df6fe178286b9070a9e8cbad6d66690a985a7285ef3758102527bd372c94d399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T17:08:05.058 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:05 smithi097 podman[62821]: 2024-04-25 17:08:05.059934042 +0000 UTC m=+0.395313063 container init df6fe178286b9070a9e8cbad6d66690a985a7285ef3758102527bd372c94d399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T17:08:05.200 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:08:05.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 -- 172.21.15.96:0/2326684744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf4103c00 msgr2=0x7f9bf4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:05.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/2326684744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf4103c00 0x7f9bf4104010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009900 tx=0x7f9be802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 -- 172.21.15.96:0/2326684744 shutdown_connections 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/2326684744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf4108900 0x7f9bf410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/2326684744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf4103c00 0x7f9bf4104010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/2326684744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bf41029c0 0x7f9bf41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 -- 172.21.15.96:0/2326684744 >> 172.21.15.96:0/2326684744 conn(0x7f9bf40fdbd0 msgr2=0x7f9bf4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 -- 172.21.15.96:0/2326684744 shutdown_connections 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.326+0000 7f9bf99d7640 1 -- 172.21.15.96:0/2326684744 wait complete. 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 Processor -- start 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 -- start start 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf41029c0 0x7f9bf4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bf4103c00 0x7f9bf4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf4108900 0x7f9bf41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:05.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf410e4e0 con 0x7f9bf41029c0 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf410e360 con 0x7f9bf4108900 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf99d7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf410e660 con 0x7f9bf4103c00 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf41029c0 0x7f9bf4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf41029c0 0x7f9bf4199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37596/0 (socket says 172.21.15.96:37596) 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 -- 172.21.15.96:0/124369136 learned_addr learned my addr 172.21.15.96:0/124369136 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf37fe640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf4108900 0x7f9bf41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 -- 172.21.15.96:0/124369136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bf4103c00 msgr2=0x7f9bf4199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bf4103c00 0x7f9bf4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 -- 172.21.15.96:0/124369136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf4108900 msgr2=0x7f9bf41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf4108900 0x7f9bf41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 -- 172.21.15.96:0/124369136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf41a0c80 con 0x7f9bf41029c0 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf37fe640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf4108900 0x7f9bf41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:05.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.327+0000 7f9bf2ffd640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf41029c0 0x7f9bf4199140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00cce0 tx=0x7f9bec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:05.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.328+0000 7f9bf89d5640 1 -- 172.21.15.96:0/124369136 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec004b20 con 0x7f9bf41029c0 2024-04-25T17:08:05.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.328+0000 7f9bf89d5640 1 -- 172.21.15.96:0/124369136 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec003e40 con 0x7f9bf41029c0 2024-04-25T17:08:05.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.328+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf41a0f70 con 0x7f9bf41029c0 2024-04-25T17:08:05.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.328+0000 7f9bf89d5640 1 -- 172.21.15.96:0/124369136 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0047b0 con 0x7f9bf41029c0 2024-04-25T17:08:05.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.328+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf41a13e0 con 0x7f9bf41029c0 2024-04-25T17:08:05.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.329+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf4107710 con 0x7f9bf41029c0 2024-04-25T17:08:05.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.330+0000 7f9bf89d5640 1 -- 172.21.15.96:0/124369136 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9bec003990 con 0x7f9bf41029c0 2024-04-25T17:08:05.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.331+0000 7f9bf89d5640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f9bdc0afaf0 0x7f9bdc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:05.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.331+0000 7f9bf27fc640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f9bdc0afaf0 0x7f9bdc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:05.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.331+0000 7f9bf89d5640 1 -- 172.21.15.96:0/124369136 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f9bec0ded10 con 0x7f9bf41029c0 2024-04-25T17:08:05.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.331+0000 7f9bf27fc640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f9bdc0afaf0 0x7f9bdc0b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9be8031500 tx=0x7f9be8031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:05.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.335+0000 7f9bf89d5640 1 -- 172.21.15.96:0/124369136 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9bec014030 con 0x7f9bf41029c0 2024-04-25T17:08:05.483 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:05 smithi097 podman[62821]: 2024-04-25 17:08:05.062414529 +0000 UTC m=+0.397793551 container start df6fe178286b9070a9e8cbad6d66690a985a7285ef3758102527bd372c94d399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T17:08:05.483 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:05 smithi097 ceph-mds[62840]: -- 172.21.15.97:0/2206603637 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2e2937a40 con 0x55d2e297f600 2024-04-25T17:08:05.483 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b[62836]: starting mds.b at 2024-04-25T17:08:05.483 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:05 smithi097 bash[62821]: df6fe178286b9070a9e8cbad6d66690a985a7285ef3758102527bd372c94d399 2024-04-25T17:08:05.484 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:05 smithi097 systemd[1]: Started Ceph mds.b for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:08:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.588+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bf402db50 con 0x7f9bf41029c0 2024-04-25T17:08:05.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.589+0000 7f9bf89d5640 1 -- 172.21.15.96:0/124369136 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f9bec0a7cf0 con 0x7f9bf41029c0 2024-04-25T17:08:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:05.591 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T17:08:05.591 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-04-25T17:08:05.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f9bdc0afaf0 msgr2=0x7f9bdc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:05.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f9bdc0afaf0 0x7f9bdc0b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9be8031500 tx=0x7f9be8031400 comp rx=0 tx=0).stop 2024-04-25T17:08:05.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf41029c0 msgr2=0x7f9bf4199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf41029c0 0x7f9bf4199140 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00cce0 tx=0x7f9bec00a620 comp rx=0 tx=0).stop 2024-04-25T17:08:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 shutdown_connections 2024-04-25T17:08:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bf4108900 0x7f9bf41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f9bdc0afaf0 0x7f9bdc0b2000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bf4103c00 0x7f9bf4199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 --2- 172.21.15.96:0/124369136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9bf41029c0 0x7f9bf4199140 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 >> 172.21.15.96:0/124369136 conn(0x7f9bf40fdbd0 msgr2=0x7f9bf41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:05.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 shutdown_connections 2024-04-25T17:08:05.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:05.591+0000 7f9bf99d7640 1 -- 172.21.15.96:0/124369136 wait complete. 2024-04-25T17:08:06.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:06.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:08:06.425 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:06 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/124369136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:06.452 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:06.452 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:06 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/124369136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.453 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:06.454 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:08:06.454 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:06 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/124369136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:07.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:07 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] up:boot 2024-04-25T17:08:07.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:07 smithi096 ceph-mon[29131]: fsmap 12 up:standby 2024-04-25T17:08:07.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:07 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:08:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:07 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] up:boot 2024-04-25T17:08:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:07 smithi097 ceph-mon[32602]: fsmap 12 up:standby 2024-04-25T17:08:07.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:07 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:08:07.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:07 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] up:boot 2024-04-25T17:08:07.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:07 smithi171 ceph-mon[33189]: fsmap 12 up:standby 2024-04-25T17:08:07.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:07 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:08:07.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:08:07.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 -- 172.21.15.96:0/2378750801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 msgr2=0x7efe8c10be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:07.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 --2- 172.21.15.96:0/2378750801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c10be50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efe84009ca0 tx=0x7efe8402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 -- 172.21.15.96:0/2378750801 shutdown_connections 2024-04-25T17:08:07.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 --2- 172.21.15.96:0/2378750801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c10be50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 --2- 172.21.15.96:0/2378750801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe8c104d30 0x7efe8c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 --2- 172.21.15.96:0/2378750801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe8c103af0 0x7efe8c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 -- 172.21.15.96:0/2378750801 >> 172.21.15.96:0/2378750801 conn(0x7efe8c0fed40 msgr2=0x7efe8c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 -- 172.21.15.96:0/2378750801 shutdown_connections 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.717+0000 7efe90a51640 1 -- 172.21.15.96:0/2378750801 wait complete. 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 Processor -- start 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 -- start start 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe8c103af0 0x7efe8c077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe8c104d30 0x7efe8c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c10f620 con 0x7efe8c109a30 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c10f4a0 con 0x7efe8c103af0 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe90a51640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c10f7a0 con 0x7efe8c104d30 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe8ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe8ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33242/0 (socket says 172.21.15.96:33242) 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.718+0000 7efe8ad9d640 1 -- 172.21.15.96:0/2457757251 learned_addr learned my addr 172.21.15.96:0/2457757251 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:07.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe8a59c640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe8c103af0 0x7efe8c077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe89d9b640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe8c104d30 0x7efe8c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe8ad9d640 1 -- 172.21.15.96:0/2457757251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe8c104d30 msgr2=0x7efe8c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe8ad9d640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe8c104d30 0x7efe8c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe8ad9d640 1 -- 172.21.15.96:0/2457757251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe8c103af0 msgr2=0x7efe8c077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe8ad9d640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe8c103af0 0x7efe8c077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe8ad9d640 1 -- 172.21.15.96:0/2457757251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c076c70 con 0x7efe8c109a30 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe8ad9d640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c076540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe840097b0 tx=0x7efe84031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe7b7fe640 1 -- 172.21.15.96:0/2457757251 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84003bf0 con 0x7efe8c109a30 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe7b7fe640 1 -- 172.21.15.96:0/2457757251 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe84003d90 con 0x7efe8c109a30 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c1a42e0 con 0x7efe8c109a30 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe7b7fe640 1 -- 172.21.15.96:0/2457757251 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8402fc30 con 0x7efe8c109a30 2024-04-25T17:08:07.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.719+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c1a4740 con 0x7efe8c109a30 2024-04-25T17:08:07.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.720+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8c06a190 con 0x7efe8c109a30 2024-04-25T17:08:07.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.722+0000 7efe7b7fe640 1 -- 172.21.15.96:0/2457757251 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7efe84004040 con 0x7efe8c109a30 2024-04-25T17:08:07.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.723+0000 7efe7b7fe640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efe740afaf0 0x7efe740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:07.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.723+0000 7efe8a59c640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efe740afaf0 0x7efe740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:07.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.723+0000 7efe7b7fe640 1 -- 172.21.15.96:0/2457757251 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7efe840fffd0 con 0x7efe8c109a30 2024-04-25T17:08:07.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.723+0000 7efe8a59c640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efe740afaf0 0x7efe740b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe8c1046f0 tx=0x7efe7c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:07.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.726+0000 7efe7b7fe640 1 -- 172.21.15.96:0/2457757251 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe840c9050 con 0x7efe8c109a30 2024-04-25T17:08:07.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.948+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe8c108840 con 0x7efe8c109a30 2024-04-25T17:08:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.949+0000 7efe7b7fe640 1 -- 172.21.15.96:0/2457757251 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7efe840c8950 con 0x7efe8c109a30 2024-04-25T17:08:07.949 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:07.949 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T17:08:07.949 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 13 2024-04-25T17:08:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efe740afaf0 msgr2=0x7efe740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efe740afaf0 0x7efe740b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe8c1046f0 tx=0x7efe7c00b390 comp rx=0 tx=0).stop 2024-04-25T17:08:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 msgr2=0x7efe8c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c076540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe840097b0 tx=0x7efe84031840 comp rx=0 tx=0).stop 2024-04-25T17:08:07.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 shutdown_connections 2024-04-25T17:08:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe8c109a30 0x7efe8c076540 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efe740afaf0 0x7efe740b2000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe8c104d30 0x7efe8c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 --2- 172.21.15.96:0/2457757251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe8c103af0 0x7efe8c077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 >> 172.21.15.96:0/2457757251 conn(0x7efe8c0fed40 msgr2=0x7efe8c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 shutdown_connections 2024-04-25T17:08:07.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:07.951+0000 7efe90a51640 1 -- 172.21.15.96:0/2457757251 wait complete. 2024-04-25T17:08:07.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 -- 172.21.15.96:0/1105016890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104103940 msgr2=0x7f9104103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 --2- 172.21.15.96:0/1105016890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104103940 0x7f9104103df0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009900 tx=0x7f90f802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 -- 172.21.15.96:0/1105016890 shutdown_connections 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 --2- 172.21.15.96:0/1105016890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9104104480 0x7f910410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 --2- 172.21.15.96:0/1105016890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104103940 0x7f9104103df0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 --2- 172.21.15.96:0/1105016890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9104102f60 0x7f9104103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 -- 172.21.15.96:0/1105016890 >> 172.21.15.96:0/1105016890 conn(0x7f91040fd360 msgr2=0x7f91040ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 -- 172.21.15.96:0/1105016890 shutdown_connections 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.074+0000 7f910bf00640 1 -- 172.21.15.96:0/1105016890 wait complete. 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 Processor -- start 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 -- start start 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104102f60 0x7f9104102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9104103940 0x7f910419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9104104480 0x7f91041a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f910410e530 con 0x7f9104102f60 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f910410e3b0 con 0x7f9104103940 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910bf00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f910410e6b0 con 0x7f9104104480 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104102f60 0x7f9104102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104102f60 0x7f9104102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33248/0 (socket says 172.21.15.96:33248) 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 -- 172.21.15.96:0/1477883104 learned_addr learned my addr 172.21.15.96:0/1477883104 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910949b640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9104103940 0x7f910419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f910a49d640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9104104480 0x7f91041a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 -- 172.21.15.96:0/1477883104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9104104480 msgr2=0x7f91041a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9104104480 0x7f91041a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 -- 172.21.15.96:0/1477883104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9104103940 msgr2=0x7f910419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9104103940 0x7f910419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 -- 172.21.15.96:0/1477883104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91041a1cf0 con 0x7f9104102f60 2024-04-25T17:08:08.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.075+0000 7f9109c9c640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104102f60 0x7f9104102220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00db20 tx=0x7f90fc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:08.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.076+0000 7f90f6ffd640 1 -- 172.21.15.96:0/1477883104 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc017070 con 0x7f9104102f60 2024-04-25T17:08:08.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.076+0000 7f90f6ffd640 1 -- 172.21.15.96:0/1477883104 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc003e20 con 0x7f9104102f60 2024-04-25T17:08:08.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.076+0000 7f90f6ffd640 1 -- 172.21.15.96:0/1477883104 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0106c0 con 0x7f9104102f60 2024-04-25T17:08:08.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.076+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91041a1fe0 con 0x7f9104102f60 2024-04-25T17:08:08.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.076+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91041a2450 con 0x7f9104102f60 2024-04-25T17:08:08.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.077+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9104107900 con 0x7f9104102f60 2024-04-25T17:08:08.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.078+0000 7f90f6ffd640 1 -- 172.21.15.96:0/1477883104 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f90fc004140 con 0x7f9104102f60 2024-04-25T17:08:08.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.079+0000 7f90f6ffd640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90f00afb70 0x7f90f00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.079+0000 7f910949b640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90f00afb70 0x7f90f00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.079+0000 7f90f6ffd640 1 -- 172.21.15.96:0/1477883104 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f90fc0de420 con 0x7f9104102f60 2024-04-25T17:08:08.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.079+0000 7f910949b640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90f00afb70 0x7f90f00b2080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009230 tx=0x7f90f8031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:08.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.084+0000 7f90f6ffd640 1 -- 172.21.15.96:0/1477883104 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90fc0a6e50 con 0x7f9104102f60 2024-04-25T17:08:08.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.280+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f910402dd20 con 0x7f9104102f60 2024-04-25T17:08:08.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.281+0000 7f90f6ffd640 1 -- 172.21.15.96:0/1477883104 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18408 (secure 0 0 0) 0x7f90fc0ab150 con 0x7f9104102f60 2024-04-25T17:08:08.281 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:08.282 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":72,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:07:22.448056+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90f00afb70 msgr2=0x7f90f00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90f00afb70 0x7f90f00b2080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009230 tx=0x7f90f8031400 comp rx=0 tx=0).stop 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104102f60 msgr2=0x7f9104102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104102f60 0x7f9104102220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00db20 tx=0x7f90fc00b260 comp rx=0 tx=0).stop 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 shutdown_connections 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90f00afb70 0x7f90f00b2080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9104104480 0x7f91041a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9104103940 0x7f910419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 --2- 172.21.15.96:0/1477883104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9104102f60 0x7f9104102220 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 >> 172.21.15.96:0/1477883104 conn(0x7f91040fd360 msgr2=0x7f9104104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 shutdown_connections 2024-04-25T17:08:08.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.284+0000 7f910bf00640 1 -- 172.21.15.96:0/1477883104 wait complete. 2024-04-25T17:08:08.294 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:08.294 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.294 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.294 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.294 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2457757251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:08.295 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:08 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:08:08.296 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 -- 172.21.15.96:0/34334627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c105090 msgr2=0x7f281c1054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 --2- 172.21.15.96:0/34334627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c105090 0x7f281c1054a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2818009900 tx=0x7f281802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 -- 172.21.15.96:0/34334627 shutdown_connections 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 --2- 172.21.15.96:0/34334627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f281c102240 0x7f281c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 --2- 172.21.15.96:0/34334627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c101740 0x7f281c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 --2- 172.21.15.96:0/34334627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c105090 0x7f281c1054a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 -- 172.21.15.96:0/34334627 >> 172.21.15.96:0/34334627 conn(0x7f281c0fd340 msgr2=0x7f281c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 -- 172.21.15.96:0/34334627 shutdown_connections 2024-04-25T17:08:08.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.412+0000 7f2823413640 1 -- 172.21.15.96:0/34334627 wait complete. 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 Processor -- start 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 -- start start 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c101740 0x7f281c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c102240 0x7f281c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f281c105090 0x7f281c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10f600 con 0x7f281c101740 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10f480 con 0x7f281c102240 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f2823413640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f281c10f780 con 0x7f281c105090 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f28211af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c101740 0x7f281c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f28211af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c101740 0x7f281c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33254/0 (socket says 172.21.15.96:33254) 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f28211af640 1 -- 172.21.15.96:0/303173077 learned_addr learned my addr 172.21.15.96:0/303173077 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f28209ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c102240 0x7f281c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.413+0000 7f28219b0640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f281c105090 0x7f281c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28211af640 1 -- 172.21.15.96:0/303173077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f281c105090 msgr2=0x7f281c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28211af640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f281c105090 0x7f281c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28211af640 1 -- 172.21.15.96:0/303173077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c102240 msgr2=0x7f281c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28211af640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c102240 0x7f281c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28211af640 1 -- 172.21.15.96:0/303173077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c1a1da0 con 0x7f281c101740 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28209ae640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c102240 0x7f281c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28211af640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c101740 0x7f281c19a260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2818006010 tx=0x7f2818004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28127fc640 1 -- 172.21.15.96:0/303173077 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281803f030 con 0x7f281c101740 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c1a2030 con 0x7f281c101740 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c1a2570 con 0x7f281c101740 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28127fc640 1 -- 172.21.15.96:0/303173077 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2818031910 con 0x7f281c101740 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.414+0000 7f28127fc640 1 -- 172.21.15.96:0/303173077 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818049060 con 0x7f281c101740 2024-04-25T17:08:08.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.415+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28000052c0 con 0x7f281c101740 2024-04-25T17:08:08.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.417+0000 7f28127fc640 1 -- 172.21.15.96:0/303173077 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2818042030 con 0x7f281c101740 2024-04-25T17:08:08.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.417+0000 7f28127fc640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28080afaf0 0x7f28080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.417+0000 7f28209ae640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28080afaf0 0x7f28080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.418+0000 7f28127fc640 1 -- 172.21.15.96:0/303173077 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f28180fece0 con 0x7f281c101740 2024-04-25T17:08:08.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.418+0000 7f28209ae640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28080afaf0 0x7f28080b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f281c19b300 tx=0x7f280c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:08.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.423+0000 7f28127fc640 1 -- 172.21.15.96:0/303173077 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28180c7710 con 0x7f281c101740 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:08:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:08:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2457757251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:08 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:08:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:08:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2457757251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:08 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.617+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2800005880 con 0x7f281c101740 2024-04-25T17:08:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.618+0000 7f28127fc640 1 -- 172.21.15.96:0/303173077 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f28180cba10 con 0x7f281c101740 2024-04-25T17:08:08.618 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28080afaf0 msgr2=0x7f28080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28080afaf0 0x7f28080b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f281c19b300 tx=0x7f280c005f90 comp rx=0 tx=0).stop 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c101740 msgr2=0x7f281c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c101740 0x7f281c19a260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2818006010 tx=0x7f2818004270 comp rx=0 tx=0).stop 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 shutdown_connections 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28080afaf0 0x7f28080b2000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f281c105090 0x7f281c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c102240 0x7f281c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 --2- 172.21.15.96:0/303173077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c101740 0x7f281c19a260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 >> 172.21.15.96:0/303173077 conn(0x7f281c0fd340 msgr2=0x7f281c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 shutdown_connections 2024-04-25T17:08:08.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.620+0000 7f2823413640 1 -- 172.21.15.96:0/303173077 wait complete. 2024-04-25T17:08:08.631 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T17:08:08.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 -- 172.21.15.96:0/4172750698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c101e60 msgr2=0x7fae6c106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 --2- 172.21.15.96:0/4172750698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c101e60 0x7fae6c106b20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae6402a410 comp rx=0 tx=0).stop 2024-04-25T17:08:08.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 -- 172.21.15.96:0/4172750698 shutdown_connections 2024-04-25T17:08:08.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 --2- 172.21.15.96:0/4172750698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae6c109780 0x7fae6c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 --2- 172.21.15.96:0/4172750698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6c102360 0x7fae6c1090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 --2- 172.21.15.96:0/4172750698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c101e60 0x7fae6c106b20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 -- 172.21.15.96:0/4172750698 >> 172.21.15.96:0/4172750698 conn(0x7fae6c0fd370 msgr2=0x7fae6c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:08.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 -- 172.21.15.96:0/4172750698 shutdown_connections 2024-04-25T17:08:08.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.739+0000 7fae72e13640 1 -- 172.21.15.96:0/4172750698 wait complete. 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 Processor -- start 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 -- start start 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae6c101e60 0x7fae6c19a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c102360 0x7fae6c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6c109780 0x7fae6c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c10f5e0 con 0x7fae6c102360 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c10f460 con 0x7fae6c109780 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae72e13640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c10f760 con 0x7fae6c101e60 2024-04-25T17:08:08.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae6bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c102360 0x7fae6c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae6bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c102360 0x7fae6c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33264/0 (socket says 172.21.15.96:33264) 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.740+0000 7fae6bfff640 1 -- 172.21.15.96:0/555951370 learned_addr learned my addr 172.21.15.96:0/555951370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae70baf640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae6c101e60 0x7fae6c19a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae713b0640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6c109780 0x7fae6c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae6bfff640 1 -- 172.21.15.96:0/555951370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae6c101e60 msgr2=0x7fae6c19a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae6bfff640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae6c101e60 0x7fae6c19a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae6bfff640 1 -- 172.21.15.96:0/555951370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6c109780 msgr2=0x7fae6c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae6bfff640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6c109780 0x7fae6c1a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae6bfff640 1 -- 172.21.15.96:0/555951370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c1a1d00 con 0x7fae6c102360 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae6bfff640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c102360 0x7fae6c19a700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ebc0 tx=0x7fae5c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae69ffb640 1 -- 172.21.15.96:0/555951370 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c010630 con 0x7fae6c102360 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae69ffb640 1 -- 172.21.15.96:0/555951370 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae5c010c80 con 0x7fae6c102360 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c1a1ff0 con 0x7fae6c102360 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae69ffb640 1 -- 172.21.15.96:0/555951370 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c018b60 con 0x7fae6c102360 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.741+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1a2460 con 0x7fae6c102360 2024-04-25T17:08:08.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.742+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c0fee90 con 0x7fae6c102360 2024-04-25T17:08:08.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.743+0000 7fae69ffb640 1 -- 172.21.15.96:0/555951370 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fae5c018d00 con 0x7fae6c102360 2024-04-25T17:08:08.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.744+0000 7fae69ffb640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae580afaf0 0x7fae580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:08.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.744+0000 7fae70baf640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae580afaf0 0x7fae580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:08.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.744+0000 7fae69ffb640 1 -- 172.21.15.96:0/555951370 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fae5c0dee10 con 0x7fae6c102360 2024-04-25T17:08:08.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.745+0000 7fae70baf640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae580afaf0 0x7fae580b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:08.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.749+0000 7fae69ffb640 1 -- 172.21.15.96:0/555951370 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae5c0a7e90 con 0x7fae6c102360 2024-04-25T17:08:08.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.942+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fae6c0637d0 con 0x7fae6c102360 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.943+0000 7fae69ffb640 1 -- 172.21.15.96:0/555951370 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fae5c0a7790 con 0x7fae6c102360 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:08:08.943 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae580afaf0 msgr2=0x7fae580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae580afaf0 0x7fae580b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64031400 comp rx=0 tx=0).stop 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c102360 msgr2=0x7fae6c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c102360 0x7fae6c19a700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00ebc0 tx=0x7fae5c00c6a0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 shutdown_connections 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae6c109780 0x7fae6c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae6c102360 0x7fae6c19a700 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae580afaf0 0x7fae580b2000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 --2- 172.21.15.96:0/555951370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae6c101e60 0x7fae6c19a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 >> 172.21.15.96:0/555951370 conn(0x7fae6c0fd370 msgr2=0x7fae6c105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 shutdown_connections 2024-04-25T17:08:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:08.945+0000 7fae72e13640 1 -- 172.21.15.96:0/555951370 wait complete. 2024-04-25T17:08:08.956 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T17:08:08.956 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T17:08:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.069+0000 7f9005af9640 1 -- 172.21.15.96:0/3178388248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90001029e0 msgr2=0x7f90001036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.069+0000 7f9005af9640 1 --2- 172.21.15.96:0/3178388248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90001029e0 0x7f90001036e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009900 tx=0x7f8ff002a410 comp rx=0 tx=0).stop 2024-04-25T17:08:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.069+0000 7f9005af9640 1 -- 172.21.15.96:0/3178388248 shutdown_connections 2024-04-25T17:08:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.069+0000 7f9005af9640 1 --2- 172.21.15.96:0/3178388248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9000108920 0x7f900010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:09.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.069+0000 7f9005af9640 1 --2- 172.21.15.96:0/3178388248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9000103c20 0x7f9000104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:09.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.069+0000 7f9005af9640 1 --2- 172.21.15.96:0/3178388248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90001029e0 0x7f90001036e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:09.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.069+0000 7f9005af9640 1 -- 172.21.15.96:0/3178388248 >> 172.21.15.96:0/3178388248 conn(0x7f90000fdbd0 msgr2=0x7f9000100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:09.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 -- 172.21.15.96:0/3178388248 shutdown_connections 2024-04-25T17:08:09.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 -- 172.21.15.96:0/3178388248 wait complete. 2024-04-25T17:08:09.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 Processor -- start 2024-04-25T17:08:09.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 -- start start 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90001029e0 0x7f9000199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9000103c20 0x7f90001999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9000108920 0x7f90001a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f900010e4e0 con 0x7f9000103c20 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f900010e360 con 0x7f90001029e0 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.070+0000 7f9005af9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f900010e660 con 0x7f9000108920 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9000103c20 0x7f90001999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9000103c20 0x7f90001999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33284/0 (socket says 172.21.15.96:33284) 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 -- 172.21.15.96:0/3607206729 learned_addr learned my addr 172.21.15.96:0/3607206729 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8fff7fe640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90001029e0 0x7f9000199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 -- 172.21.15.96:0/3607206729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9000108920 msgr2=0x7f90001a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9000108920 0x7f90001a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 -- 172.21.15.96:0/3607206729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90001029e0 msgr2=0x7f9000199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90001029e0 0x7f9000199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:09.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 -- 172.21.15.96:0/3607206729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90001a0f20 con 0x7f9000103c20 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ff6ffd640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9000103c20 0x7f90001999d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800d770 tx=0x7f8fe800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ffd7fa640 1 -- 172.21.15.96:0/3607206729 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8017070 con 0x7f9000103c20 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ffd7fa640 1 -- 172.21.15.96:0/3607206729 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe80054d0 con 0x7f9000103c20 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90001a1210 con 0x7f9000103c20 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f8ffd7fa640 1 -- 172.21.15.96:0/3607206729 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8004c30 con 0x7f9000103c20 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.071+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001a1710 con 0x7f9000103c20 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.072+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9000107730 con 0x7f9000103c20 2024-04-25T17:08:09.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.074+0000 7f8ffd7fa640 1 -- 172.21.15.96:0/3607206729 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8fe8004dd0 con 0x7f9000103c20 2024-04-25T17:08:09.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.075+0000 7f8ffd7fa640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fec0afaf0 0x7f8fec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:09.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.075+0000 7f8fff7fe640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fec0afaf0 0x7f8fec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:09.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.075+0000 7f8ffd7fa640 1 -- 172.21.15.96:0/3607206729 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8fe80de650 con 0x7f9000103c20 2024-04-25T17:08:09.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.075+0000 7f8fff7fe640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fec0afaf0 0x7f8fec0b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005ab0 tx=0x7f8ff0031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:09.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.079+0000 7f8ffd7fa640 1 -- 172.21.15.96:0/3607206729 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fe80a76d0 con 0x7f9000103c20 2024-04-25T17:08:09.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:09 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1477883104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:09.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:09 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/303173077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:08:09.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:09 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/555951370' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T17:08:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:09.260+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f900006c2a0 con 0x7f9000103c20 2024-04-25T17:08:09.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:09 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1477883104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:09.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:09 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/303173077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:08:09.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:09 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/555951370' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T17:08:09.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:09 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1477883104' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:09.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:09 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/303173077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:08:09.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:09 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/555951370' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T17:08:10.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.127+0000 7f8ffd7fa640 1 -- 172.21.15.96:0/3607206729 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f8fe80a6fd0 con 0x7f9000103c20 2024-04-25T17:08:10.127 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-04-25T17:08:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fec0afaf0 msgr2=0x7f8fec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fec0afaf0 0x7f8fec0b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005ab0 tx=0x7f8ff0031400 comp rx=0 tx=0).stop 2024-04-25T17:08:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9000103c20 msgr2=0x7f90001999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9000103c20 0x7f90001999d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800d770 tx=0x7f8fe800dc30 comp rx=0 tx=0).stop 2024-04-25T17:08:10.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 shutdown_connections 2024-04-25T17:08:10.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8fec0afaf0 0x7f8fec0b2000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9000108920 0x7f90001a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9000103c20 0x7f90001999d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 --2- 172.21.15.96:0/3607206729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90001029e0 0x7f9000199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 >> 172.21.15.96:0/3607206729 conn(0x7f90000fdbd0 msgr2=0x7f9000104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:10.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 shutdown_connections 2024-04-25T17:08:10.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.130+0000 7f9005af9640 1 -- 172.21.15.96:0/3607206729 wait complete. 2024-04-25T17:08:10.143 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T17:08:10.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 -- 172.21.15.96:0/3737231854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89441043a0 msgr2=0x7f8944102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:10.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 --2- 172.21.15.96:0/3737231854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89441043a0 0x7f8944102da0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f892c009900 tx=0x7f892c02a410 comp rx=0 tx=0).stop 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 -- 172.21.15.96:0/3737231854 shutdown_connections 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 --2- 172.21.15.96:0/3737231854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89441089f0 0x7f894410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 --2- 172.21.15.96:0/3737231854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89441043a0 0x7f8944102da0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 --2- 172.21.15.96:0/3737231854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8944103ec0 0x7f89441027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 -- 172.21.15.96:0/3737231854 >> 172.21.15.96:0/3737231854 conn(0x7f89440fd320 msgr2=0x7f89440ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 -- 172.21.15.96:0/3737231854 shutdown_connections 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.257+0000 7f8948c61640 1 -- 172.21.15.96:0/3737231854 wait complete. 2024-04-25T17:08:10.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 Processor -- start 2024-04-25T17:08:10.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 -- start start 2024-04-25T17:08:10.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8944103ec0 0x7f8944199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:10.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89441089f0 0x7f894419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f894419f2c0 0x7f89441a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f894410e500 con 0x7f8944103ec0 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f894410e380 con 0x7f89441089f0 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.258+0000 7f8948c61640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f894410e680 con 0x7f894419f2c0 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8944103ec0 0x7f8944199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8944103ec0 0x7f8944199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33290/0 (socket says 172.21.15.96:33290) 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 -- 172.21.15.96:0/1773902304 learned_addr learned my addr 172.21.15.96:0/1773902304 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f8939d9b640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89441089f0 0x7f894419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 -- 172.21.15.96:0/1773902304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f894419f2c0 msgr2=0x7f89441a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f8942d9d640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f894419f2c0 0x7f89441a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f894419f2c0 0x7f89441a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 -- 172.21.15.96:0/1773902304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89441089f0 msgr2=0x7f894419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89441089f0 0x7f894419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:10.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 -- 172.21.15.96:0/1773902304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89441a1d50 con 0x7f8944103ec0 2024-04-25T17:08:10.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f894259c640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8944103ec0 0x7f8944199ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f893400d740 tx=0x7f893400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:10.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f893bfff640 1 -- 172.21.15.96:0/1773902304 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934017070 con 0x7f8944103ec0 2024-04-25T17:08:10.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f893bfff640 1 -- 172.21.15.96:0/1773902304 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f893400de50 con 0x7f8944103ec0 2024-04-25T17:08:10.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89441a1fe0 con 0x7f8944103ec0 2024-04-25T17:08:10.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f893bfff640 1 -- 172.21.15.96:0/1773902304 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934004ce0 con 0x7f8944103ec0 2024-04-25T17:08:10.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.259+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89441a2450 con 0x7f8944103ec0 2024-04-25T17:08:10.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.260+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89441a26b0 con 0x7f8944103ec0 2024-04-25T17:08:10.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.263+0000 7f893bfff640 1 -- 172.21.15.96:0/1773902304 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8934004e80 con 0x7f8944103ec0 2024-04-25T17:08:10.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.263+0000 7f893bfff640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f89300afaf0 0x7f89300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:10.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.263+0000 7f8939d9b640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f89300afaf0 0x7f89300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:10.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.263+0000 7f893bfff640 1 -- 172.21.15.96:0/1773902304 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7497+0+0 (secure 0 0 0) 0x7f89340de640 con 0x7f8944103ec0 2024-04-25T17:08:10.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.264+0000 7f8939d9b640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f89300afaf0 0x7f89300b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f892c009230 tx=0x7f892c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:10.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.267+0000 7f893bfff640 1 -- 172.21.15.96:0/1773902304 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89340a7550 con 0x7f8944103ec0 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3607206729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:10.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:08:10.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:10.448+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f89441a2da0 con 0x7f8944103ec0 2024-04-25T17:08:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3607206729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T17:08:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:08:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3607206729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T17:08:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:08:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:08:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:08:11.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.133+0000 7f893bfff640 1 -- 172.21.15.96:0/1773902304 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) ==== 170+0+0 (secure 0 0 0) 0x7f89340a6e50 con 0x7f8944103ec0 2024-04-25T17:08:11.133 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-04-25T17:08:11.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f89300afaf0 msgr2=0x7f89300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:11.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f89300afaf0 0x7f89300b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f892c009230 tx=0x7f892c031400 comp rx=0 tx=0).stop 2024-04-25T17:08:11.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8944103ec0 msgr2=0x7f8944199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:11.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8944103ec0 0x7f8944199ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f893400d740 tx=0x7f893400dc00 comp rx=0 tx=0).stop 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 shutdown_connections 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f89300afaf0 0x7f89300b2000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f894419f2c0 0x7f89441a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.136+0000 7f8948c61640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89441089f0 0x7f894419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.137+0000 7f8948c61640 1 --2- 172.21.15.96:0/1773902304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8944103ec0 0x7f8944199ea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.137+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 >> 172.21.15.96:0/1773902304 conn(0x7f89440fd320 msgr2=0x7f8944109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.137+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 shutdown_connections 2024-04-25T17:08:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.137+0000 7f8948c61640 1 -- 172.21.15.96:0/1773902304 wait complete. 2024-04-25T17:08:11.156 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T17:08:11.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.268+0000 7f50a0c49640 1 -- 172.21.15.96:0/1237572624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 msgr2=0x7f509c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.268+0000 7f50a0c49640 1 --2- 172.21.15.96:0/1237572624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c102e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f508c009900 tx=0x7f508c02a410 comp rx=0 tx=0).stop 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 -- 172.21.15.96:0/1237572624 shutdown_connections 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 --2- 172.21.15.96:0/1237572624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f509c1080d0 0x7f509c10a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 --2- 172.21.15.96:0/1237572624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509c1033d0 0x7f509c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 --2- 172.21.15.96:0/1237572624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c102e90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 -- 172.21.15.96:0/1237572624 >> 172.21.15.96:0/1237572624 conn(0x7f509c0fd380 msgr2=0x7f509c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 -- 172.21.15.96:0/1237572624 shutdown_connections 2024-04-25T17:08:11.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 -- 172.21.15.96:0/1237572624 wait complete. 2024-04-25T17:08:11.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 Processor -- start 2024-04-25T17:08:11.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.269+0000 7f50a0c49640 1 -- start start 2024-04-25T17:08:11.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:11.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f509c1033d0 0x7f509c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509c1080d0 0x7f509c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f509c10d420 con 0x7f509c102190 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f509c10d2a0 con 0x7f509c1080d0 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f509c10d5a0 con 0x7f509c1033d0 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33316/0 (socket says 172.21.15.96:33316) 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 -- 172.21.15.96:0/3787389736 learned_addr learned my addr 172.21.15.96:0/3787389736 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 -- 172.21.15.96:0/3787389736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f509c1033d0 msgr2=0x7f509c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f509c1033d0 0x7f509c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 -- 172.21.15.96:0/3787389736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509c1080d0 msgr2=0x7f509c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509ad9d640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509c1080d0 0x7f509c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509c1080d0 0x7f509c1a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:11.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 -- 172.21.15.96:0/3787389736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c1a1ce0 con 0x7f509c102190 2024-04-25T17:08:11.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f509a59c640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c19a250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:11.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f508b7fe640 1 -- 172.21.15.96:0/3787389736 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c03f030 con 0x7f509c102190 2024-04-25T17:08:11.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f508b7fe640 1 -- 172.21.15.96:0/3787389736 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f508c003aa0 con 0x7f509c102190 2024-04-25T17:08:11.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509c1a1f70 con 0x7f509c102190 2024-04-25T17:08:11.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f508b7fe640 1 -- 172.21.15.96:0/3787389736 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c035070 con 0x7f509c102190 2024-04-25T17:08:11.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.270+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509c1a23d0 con 0x7f509c102190 2024-04-25T17:08:11.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.272+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509c1a2600 con 0x7f509c102190 2024-04-25T17:08:11.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.273+0000 7f508b7fe640 1 -- 172.21.15.96:0/3787389736 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f508c042030 con 0x7f509c102190 2024-04-25T17:08:11.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.273+0000 7f508b7fe640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f50840afaf0 0x7f50840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:11.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.273+0000 7f5099d9b640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f50840afaf0 0x7f50840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:11.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.273+0000 7f508b7fe640 1 -- 172.21.15.96:0/3787389736 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7f508c0ff090 con 0x7f509c102190 2024-04-25T17:08:11.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.274+0000 7f5099d9b640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f50840afaf0 0x7f50840b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f509c19b2f0 tx=0x7f5090005c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.278+0000 7f508b7fe640 1 -- 172.21.15.96:0/3787389736 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f508c0c7e20 con 0x7f509c102190 2024-04-25T17:08:11.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:11 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3607206729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T17:08:11.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:11 smithi096 ceph-mon[29131]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T17:08:11.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:11 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1773902304' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T17:08:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:11 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3607206729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T17:08:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:11 smithi097 ceph-mon[32602]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T17:08:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:11 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1773902304' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T17:08:11.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:11.493+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f509c1a27d0 con 0x7f509c102190 2024-04-25T17:08:11.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:11 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3607206729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T17:08:11.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:11 smithi171 ceph-mon[33189]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T17:08:11.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:11 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1773902304' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T17:08:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.136+0000 7f508b7fe640 1 -- 172.21.15.96:0/3787389736 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f508c0c7720 con 0x7f509c102190 2024-04-25T17:08:12.135 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f50840afaf0 msgr2=0x7f50840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f50840afaf0 0x7f50840b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f509c19b2f0 tx=0x7f5090005c70 comp rx=0 tx=0).stop 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 msgr2=0x7f509c19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c19a250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f508c006010 tx=0x7f508c031a50 comp rx=0 tx=0).stop 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 shutdown_connections 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509c1080d0 0x7f509c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f50840afaf0 0x7f50840b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f509c1033d0 0x7f509c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 --2- 172.21.15.96:0/3787389736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f509c102190 0x7f509c19a250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 >> 172.21.15.96:0/3787389736 conn(0x7f509c0fd380 msgr2=0x7f509c1060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 shutdown_connections 2024-04-25T17:08:12.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.138+0000 7f50a0c49640 1 -- 172.21.15.96:0/3787389736 wait complete. 2024-04-25T17:08:12.157 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:08:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 -- 172.21.15.96:0/2631866125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c104d40 msgr2=0x7ff05c105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 --2- 172.21.15.96:0/2631866125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c104d40 0x7ff05c105150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff058009900 tx=0x7ff05802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 -- 172.21.15.96:0/2631866125 shutdown_connections 2024-04-25T17:08:12.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 --2- 172.21.15.96:0/2631866125 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff05c109a40 0x7ff05c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 --2- 172.21.15.96:0/2631866125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c104d40 0x7ff05c105150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 --2- 172.21.15.96:0/2631866125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05c103b00 0x7ff05c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 -- 172.21.15.96:0/2631866125 >> 172.21.15.96:0/2631866125 conn(0x7ff05c0fed10 msgr2=0x7ff05c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 -- 172.21.15.96:0/2631866125 shutdown_connections 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.273+0000 7ff063867640 1 -- 172.21.15.96:0/2631866125 wait complete. 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 Processor -- start 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 -- start start 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05c103b00 0x7ff05c077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff05c104d40 0x7ff05c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c109a40 0x7ff05c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c10f600 con 0x7ff05c109a40 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c10f480 con 0x7ff05c103b00 2024-04-25T17:08:12.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff063867640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c10f780 con 0x7ff05c104d40 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff061e04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c109a40 0x7ff05c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff061e04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c109a40 0x7ff05c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33338/0 (socket says 172.21.15.96:33338) 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.274+0000 7ff061e04640 1 -- 172.21.15.96:0/1424667023 learned_addr learned my addr 172.21.15.96:0/1424667023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff060e02640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff05c104d40 0x7ff05c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061603640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05c103b00 0x7ff05c077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061e04640 1 -- 172.21.15.96:0/1424667023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff05c104d40 msgr2=0x7ff05c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061e04640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff05c104d40 0x7ff05c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061e04640 1 -- 172.21.15.96:0/1424667023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05c103b00 msgr2=0x7ff05c077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061e04640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05c103b00 0x7ff05c077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061e04640 1 -- 172.21.15.96:0/1424667023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c076c70 con 0x7ff05c109a40 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061603640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05c103b00 0x7ff05c077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff061e04640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c109a40 0x7ff05c076540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff054009fd0 tx=0x7ff05400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff0527fc640 1 -- 172.21.15.96:0/1424667023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054017070 con 0x7ff05c109a40 2024-04-25T17:08:12.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff0527fc640 1 -- 172.21.15.96:0/1424667023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff054003e20 con 0x7ff05c109a40 2024-04-25T17:08:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c1a42f0 con 0x7ff05c109a40 2024-04-25T17:08:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff0527fc640 1 -- 172.21.15.96:0/1424667023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054010460 con 0x7ff05c109a40 2024-04-25T17:08:12.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.275+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1a47b0 con 0x7ff05c109a40 2024-04-25T17:08:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.276+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05c108850 con 0x7ff05c109a40 2024-04-25T17:08:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.277+0000 7ff0527fc640 1 -- 172.21.15.96:0/1424667023 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff054003970 con 0x7ff05c109a40 2024-04-25T17:08:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.278+0000 7ff0527fc640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0480afb70 0x7ff0480b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.278+0000 7ff061603640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0480afb70 0x7ff0480b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.278+0000 7ff0527fc640 1 -- 172.21.15.96:0/1424667023 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff0540de720 con 0x7ff05c109a40 2024-04-25T17:08:12.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.278+0000 7ff061603640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0480afb70 0x7ff0480b2080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005d00 tx=0x7ff04c005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:12.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.282+0000 7ff0527fc640 1 -- 172.21.15.96:0/1424667023 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0540a74d0 con 0x7ff05c109a40 2024-04-25T17:08:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:12 smithi096 ceph-mon[29131]: pgmap v277: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:12 smithi096 ceph-mon[29131]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T17:08:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:12 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1773902304' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T17:08:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:12 smithi096 ceph-mon[29131]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T17:08:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:12 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3787389736' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T17:08:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:12 smithi096 ceph-mon[29131]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T17:08:12.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.462+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff05c0637a0 con 0x7ff05c109a40 2024-04-25T17:08:12.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.463+0000 7ff0527fc640 1 -- 172.21.15.96:0/1424667023 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22184 (secure 0 0 0) 0x7ff0540a6dd0 con 0x7ff05c109a40 2024-04-25T17:08:12.463 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:12.464 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:08:11.494656+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:08:12.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0480afb70 msgr2=0x7ff0480b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0480afb70 0x7ff0480b2080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005d00 tx=0x7ff04c005ef0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c109a40 msgr2=0x7ff05c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c109a40 0x7ff05c076540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff054009fd0 tx=0x7ff05400b260 comp rx=0 tx=0).stop 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 shutdown_connections 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff05c109a40 0x7ff05c076540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0480afb70 0x7ff0480b2080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff05c104d40 0x7ff05c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 --2- 172.21.15.96:0/1424667023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff05c103b00 0x7ff05c077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 >> 172.21.15.96:0/1424667023 conn(0x7ff05c0fed10 msgr2=0x7ff05c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 shutdown_connections 2024-04-25T17:08:12.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.466+0000 7ff063867640 1 -- 172.21.15.96:0/1424667023 wait complete. 2024-04-25T17:08:12.477 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:08:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:12 smithi097 ceph-mon[32602]: pgmap v277: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:12 smithi097 ceph-mon[32602]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T17:08:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:12 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1773902304' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T17:08:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:12 smithi097 ceph-mon[32602]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T17:08:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:12 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3787389736' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T17:08:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:12 smithi097 ceph-mon[32602]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T17:08:12.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:12 smithi171 ceph-mon[33189]: pgmap v277: 33 pgs: 9 creating+peering, 23 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:08:12.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:12 smithi171 ceph-mon[33189]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T17:08:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:12 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1773902304' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T17:08:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:12 smithi171 ceph-mon[33189]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T17:08:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:12 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3787389736' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T17:08:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:12 smithi171 ceph-mon[33189]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2962687625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b41029e0 msgr2=0x7fe4b41036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2962687625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b41029e0 0x7fe4b41036e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009900 tx=0x7fe4b002a410 comp rx=0 tx=0).stop 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2962687625 shutdown_connections 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2962687625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b4108920 0x7fe4b410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2962687625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b4103c20 0x7fe4b4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2962687625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b41029e0 0x7fe4b41036e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2962687625 >> 172.21.15.96:0/2962687625 conn(0x7fe4b40fdbf0 msgr2=0x7fe4b4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2962687625 shutdown_connections 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.585+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2962687625 wait complete. 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 Processor -- start 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 -- start start 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b41029e0 0x7fe4b419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b4103c20 0x7fe4b419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b4108920 0x7fe4b41a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b410f5d0 con 0x7fe4b4103c20 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b410f450 con 0x7fe4b4108920 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4bc0e7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b410f750 con 0x7fe4b41029e0 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b4103c20 0x7fe4b419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b4103c20 0x7fe4b419a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33366/0 (socket says 172.21.15.96:33366) 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 -- 172.21.15.96:0/2543794919 learned_addr learned my addr 172.21.15.96:0/2543794919 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9e83640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b41029e0 0x7fe4b419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4ba684640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b4108920 0x7fe4b41a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 -- 172.21.15.96:0/2543794919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b41029e0 msgr2=0x7fe4b419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b41029e0 0x7fe4b419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 -- 172.21.15.96:0/2543794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b4108920 msgr2=0x7fe4b41a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b4108920 0x7fe4b41a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 -- 172.21.15.96:0/2543794919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b41a1dd0 con 0x7fe4b4103c20 2024-04-25T17:08:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4ba684640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b4108920 0x7fe4b41a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.586+0000 7fe4b9682640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b4103c20 0x7fe4b419a7d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400d950 tx=0x7fe4a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.587+0000 7fe4aaffd640 1 -- 172.21.15.96:0/2543794919 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4017030 con 0x7fe4b4103c20 2024-04-25T17:08:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.587+0000 7fe4aaffd640 1 -- 172.21.15.96:0/2543794919 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4a4004020 con 0x7fe4b4103c20 2024-04-25T17:08:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.587+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b41a20c0 con 0x7fe4b4103c20 2024-04-25T17:08:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.587+0000 7fe4aaffd640 1 -- 172.21.15.96:0/2543794919 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40047a0 con 0x7fe4b4103c20 2024-04-25T17:08:12.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.587+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b41a2610 con 0x7fe4b4103c20 2024-04-25T17:08:12.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.588+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b4107730 con 0x7fe4b4103c20 2024-04-25T17:08:12.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.590+0000 7fe4aaffd640 1 -- 172.21.15.96:0/2543794919 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe4a4009a30 con 0x7fe4b4103c20 2024-04-25T17:08:12.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.590+0000 7fe4aaffd640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe4a00afaf0 0x7fe4a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.590+0000 7fe4b9e83640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe4a00afaf0 0x7fe4a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.591+0000 7fe4aaffd640 1 -- 172.21.15.96:0/2543794919 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fe4a40dec50 con 0x7fe4b4103c20 2024-04-25T17:08:12.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.591+0000 7fe4b9e83640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe4a00afaf0 0x7fe4a00b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009230 tx=0x7fe4b0031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:12.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.594+0000 7fe4aaffd640 1 -- 172.21.15.96:0/2543794919 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4a40a7980 con 0x7fe4b4103c20 2024-04-25T17:08:12.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.772+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4b402dd20 con 0x7fe4b4103c20 2024-04-25T17:08:12.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.773+0000 7fe4aaffd640 1 -- 172.21.15.96:0/2543794919 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22184 (secure 0 0 0) 0x7fe4a40a7280 con 0x7fe4b4103c20 2024-04-25T17:08:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:12.773 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":75,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:08:11.494656+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:08:12.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe4a00afaf0 msgr2=0x7fe4a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe4a00afaf0 0x7fe4a00b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009230 tx=0x7fe4b0031400 comp rx=0 tx=0).stop 2024-04-25T17:08:12.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b4103c20 msgr2=0x7fe4b419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b4103c20 0x7fe4b419a7d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400d950 tx=0x7fe4a400de10 comp rx=0 tx=0).stop 2024-04-25T17:08:12.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 shutdown_connections 2024-04-25T17:08:12.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4b4108920 0x7fe4b41a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4b4103c20 0x7fe4b419a7d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe4a00afaf0 0x7fe4a00b2000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 --2- 172.21.15.96:0/2543794919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b41029e0 0x7fe4b419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 >> 172.21.15.96:0/2543794919 conn(0x7fe4b40fdbf0 msgr2=0x7fe4b4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:12.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 shutdown_connections 2024-04-25T17:08:12.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.775+0000 7fe4bc0e7640 1 -- 172.21.15.96:0/2543794919 wait complete. 2024-04-25T17:08:12.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T17:08:12.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 -- 172.21.15.96:0/1975224021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c103960 msgr2=0x7fb86c103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 --2- 172.21.15.96:0/1975224021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c103960 0x7fb86c103df0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb860009900 tx=0x7fb86002a410 comp rx=0 tx=0).stop 2024-04-25T17:08:12.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 -- 172.21.15.96:0/1975224021 shutdown_connections 2024-04-25T17:08:12.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 --2- 172.21.15.96:0/1975224021 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb86c104480 0x7fb86c10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 --2- 172.21.15.96:0/1975224021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c103960 0x7fb86c103df0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 --2- 172.21.15.96:0/1975224021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86c102f80 0x7fb86c103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 -- 172.21.15.96:0/1975224021 >> 172.21.15.96:0/1975224021 conn(0x7fb86c0fd380 msgr2=0x7fb86c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 -- 172.21.15.96:0/1975224021 shutdown_connections 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.897+0000 7fb873213640 1 -- 172.21.15.96:0/1975224021 wait complete. 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 Processor -- start 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 -- start start 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86c102f80 0x7fb86c102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb86c103960 0x7fb86c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c104480 0x7fb86c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10e550 con 0x7fb86c104480 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10e3d0 con 0x7fb86c102f80 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb873213640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10e6d0 con 0x7fb86c103960 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb8717b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c104480 0x7fb86c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb8717b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c104480 0x7fb86c1a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33392/0 (socket says 172.21.15.96:33392) 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.898+0000 7fb8717b0640 1 -- 172.21.15.96:0/609592094 learned_addr learned my addr 172.21.15.96:0/609592094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb870faf640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86c102f80 0x7fb86c102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb86bfff640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb86c103960 0x7fb86c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb8717b0640 1 -- 172.21.15.96:0/609592094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb86c103960 msgr2=0x7fb86c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb8717b0640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb86c103960 0x7fb86c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb8717b0640 1 -- 172.21.15.96:0/609592094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86c102f80 msgr2=0x7fb86c102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb8717b0640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86c102f80 0x7fb86c102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb8717b0640 1 -- 172.21.15.96:0/609592094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c1a1cf0 con 0x7fb86c104480 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb8717b0640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c104480 0x7fb86c1a14d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eaa0 tx=0x7fb86400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb869ffb640 1 -- 172.21.15.96:0/609592094 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86400cdb0 con 0x7fb86c104480 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb869ffb640 1 -- 172.21.15.96:0/609592094 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb864005020 con 0x7fb86c104480 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb869ffb640 1 -- 172.21.15.96:0/609592094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8640116d0 con 0x7fb86c104480 2024-04-25T17:08:12.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c1a1fe0 con 0x7fb86c104480 2024-04-25T17:08:12.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.899+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c1a24a0 con 0x7fb86c104480 2024-04-25T17:08:12.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.900+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c107920 con 0x7fb86c104480 2024-04-25T17:08:12.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.901+0000 7fb869ffb640 1 -- 172.21.15.96:0/609592094 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb8640051d0 con 0x7fb86c104480 2024-04-25T17:08:12.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.902+0000 7fb869ffb640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb8540afb70 0x7fb8540b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:12.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.902+0000 7fb870faf640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb8540afb70 0x7fb8540b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:12.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.902+0000 7fb869ffb640 1 -- 172.21.15.96:0/609592094 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fb8640decb0 con 0x7fb86c104480 2024-04-25T17:08:12.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.902+0000 7fb870faf640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb8540afb70 0x7fb8540b2080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0058c0 tx=0x7fb85c00a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:12.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:12.907+0000 7fb869ffb640 1 -- 172.21.15.96:0/609592094 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb864016030 con 0x7fb86c104480 2024-04-25T17:08:13.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.123+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb86c02db50 con 0x7fb86c104480 2024-04-25T17:08:13.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.157+0000 7fb869ffb640 1 -- 172.21.15.96:0/609592094 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fb8640a79e0 con 0x7fb86c104480 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb8540afb70 msgr2=0x7fb8540b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb8540afb70 0x7fb8540b2080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0058c0 tx=0x7fb85c00a4c0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c104480 msgr2=0x7fb86c1a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c104480 0x7fb86c1a14d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eaa0 tx=0x7fb86400ef60 comp rx=0 tx=0).stop 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 shutdown_connections 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb86c104480 0x7fb86c1a14d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb8540afb70 0x7fb8540b2080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb86c103960 0x7fb86c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 --2- 172.21.15.96:0/609592094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb86c102f80 0x7fb86c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 >> 172.21.15.96:0/609592094 conn(0x7fb86c0fd380 msgr2=0x7fb86c104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:13.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 shutdown_connections 2024-04-25T17:08:13.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.159+0000 7fb873213640 1 -- 172.21.15.96:0/609592094 wait complete. 2024-04-25T17:08:13.170 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T17:08:13.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T17:08:13.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/3490646879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 msgr2=0x7f1a8810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:13.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/3490646879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8810bcb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009ca0 tx=0x7f1a8002a8c0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/3490646879 shutdown_connections 2024-04-25T17:08:13.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/3490646879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8810bcb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/3490646879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a88101750 0x7f1a88101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/3490646879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a881050a0 0x7f1a881054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/3490646879 >> 172.21.15.96:0/3490646879 conn(0x7f1a880fd350 msgr2=0x7f1a880ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/3490646879 shutdown_connections 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.281+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/3490646879 wait complete. 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 Processor -- start 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 -- start start 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a88101750 0x7f1a8819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a881050a0 0x7f1a881a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8810f610 con 0x7f1a88102250 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8810f490 con 0x7f1a881050a0 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a8d3fe640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8810f790 con 0x7f1a88101750 2024-04-25T17:08:13.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33396/0 (socket says 172.21.15.96:33396) 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 -- 172.21.15.96:0/2929647922 learned_addr learned my addr 172.21.15.96:0/2929647922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a86ffd640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a88101750 0x7f1a8819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a877fe640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a881050a0 0x7f1a881a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 -- 172.21.15.96:0/2929647922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a88101750 msgr2=0x7f1a8819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a88101750 0x7f1a8819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 -- 172.21.15.96:0/2929647922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a881050a0 msgr2=0x7f1a881a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a881050a0 0x7f1a881a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.282+0000 7f1a867fc640 1 -- 172.21.15.96:0/2929647922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a881a1b80 con 0x7f1a88102250 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.283+0000 7f1a867fc640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8819a630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c920 tx=0x7f1a7c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:13.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.283+0000 7f1a6ffff640 1 -- 172.21.15.96:0/2929647922 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c007c70 con 0x7f1a88102250 2024-04-25T17:08:13.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.283+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a881a1e70 con 0x7f1a88102250 2024-04-25T17:08:13.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.283+0000 7f1a6ffff640 1 -- 172.21.15.96:0/2929647922 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7c007e10 con 0x7f1a88102250 2024-04-25T17:08:13.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.283+0000 7f1a6ffff640 1 -- 172.21.15.96:0/2929647922 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c018820 con 0x7f1a88102250 2024-04-25T17:08:13.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.284+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a881a2330 con 0x7f1a88102250 2024-04-25T17:08:13.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.285+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8810aa40 con 0x7f1a88102250 2024-04-25T17:08:13.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.285+0000 7f1a6ffff640 1 -- 172.21.15.96:0/2929647922 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1a7c00f450 con 0x7f1a88102250 2024-04-25T17:08:13.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.286+0000 7f1a6ffff640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a700afaf0 0x7f1a700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:13.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.286+0000 7f1a86ffd640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a700afaf0 0x7f1a700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:13.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.286+0000 7f1a6ffff640 1 -- 172.21.15.96:0/2929647922 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f1a7c0decd0 con 0x7f1a88102250 2024-04-25T17:08:13.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.287+0000 7f1a86ffd640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a700afaf0 0x7f1a700b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a780058c0 tx=0x7f1a78005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:13.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.291+0000 7f1a6ffff640 1 -- 172.21.15.96:0/2929647922 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a7c014030 con 0x7f1a88102250 2024-04-25T17:08:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3787389736' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T17:08:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T17:08:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T17:08:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T17:08:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1424667023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2543794919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: pgmap v280: 97 pgs: 18 creating+peering, 48 unknown, 31 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T17:08:13.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/609592094' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T17:08:13.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:13 smithi096 ceph-mon[29131]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T17:08:13.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3787389736' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T17:08:13.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T17:08:13.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T17:08:13.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T17:08:13.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1424667023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:13.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2543794919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:13.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: pgmap v280: 97 pgs: 18 creating+peering, 48 unknown, 31 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T17:08:13.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/609592094' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T17:08:13.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:13 smithi097 ceph-mon[32602]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T17:08:13.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:13.512+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f1a8802dd20 con 0x7f1a88102250 2024-04-25T17:08:13.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3787389736' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T17:08:13.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T17:08:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T17:08:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T17:08:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1424667023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2543794919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: pgmap v280: 97 pgs: 18 creating+peering, 48 unknown, 31 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T17:08:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/609592094' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T17:08:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:13 smithi171 ceph-mon[33189]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T17:08:14.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.379+0000 7f1a6ffff640 1 -- 172.21.15.96:0/2929647922 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f1a7c0a7a00 con 0x7f1a88102250 2024-04-25T17:08:14.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a700afaf0 msgr2=0x7f1a700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:14.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a700afaf0 0x7f1a700b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a780058c0 tx=0x7f1a78005850 comp rx=0 tx=0).stop 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 msgr2=0x7f1a8819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8819a630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c920 tx=0x7f1a7c00cde0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 shutdown_connections 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a881050a0 0x7f1a881a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a88102250 0x7f1a8819a630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a700afaf0 0x7f1a700b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 --2- 172.21.15.96:0/2929647922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a88101750 0x7f1a8819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.382+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 >> 172.21.15.96:0/2929647922 conn(0x7f1a880fd350 msgr2=0x7f1a88107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.383+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 shutdown_connections 2024-04-25T17:08:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.383+0000 7f1a8d3fe640 1 -- 172.21.15.96:0/2929647922 wait complete. 2024-04-25T17:08:14.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T17:08:14.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:14 smithi096 ceph-mon[29131]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T17:08:14.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:14 smithi096 ceph-mon[29131]: Cluster is now healthy 2024-04-25T17:08:14.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:14 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/609592094' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T17:08:14.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:14 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] up:active 2024-04-25T17:08:14.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:14 smithi096 ceph-mon[29131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:08:14.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:14 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2929647922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T17:08:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:14 smithi097 ceph-mon[32602]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T17:08:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:14 smithi097 ceph-mon[32602]: Cluster is now healthy 2024-04-25T17:08:14.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:14 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/609592094' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T17:08:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:14 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] up:active 2024-04-25T17:08:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:14 smithi097 ceph-mon[32602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:08:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:14 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2929647922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T17:08:14.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 -- 172.21.15.96:0/1825908886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c108930 msgr2=0x7eff9c10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 --2- 172.21.15.96:0/1825908886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c108930 0x7eff9c10ad50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009ca0 tx=0x7eff8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 -- 172.21.15.96:0/1825908886 shutdown_connections 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 --2- 172.21.15.96:0/1825908886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c108930 0x7eff9c10ad50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 --2- 172.21.15.96:0/1825908886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff9c103c30 0x7eff9c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 --2- 172.21.15.96:0/1825908886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c1029f0 0x7eff9c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 -- 172.21.15.96:0/1825908886 >> 172.21.15.96:0/1825908886 conn(0x7eff9c0fdbc0 msgr2=0x7eff9c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 -- 172.21.15.96:0/1825908886 shutdown_connections 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.512+0000 7effa45e3640 1 -- 172.21.15.96:0/1825908886 wait complete. 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 Processor -- start 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 -- start start 2024-04-25T17:08:14.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c1029f0 0x7eff9c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c103c30 0x7eff9c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff9c108930 0x7eff9c1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c10f610 con 0x7eff9c103c30 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c10f490 con 0x7eff9c1029f0 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa45e3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c10f790 con 0x7eff9c108930 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa1b7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c103c30 0x7eff9c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa1b7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c103c30 0x7eff9c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33422/0 (socket says 172.21.15.96:33422) 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa1b7e640 1 -- 172.21.15.96:0/1178433741 learned_addr learned my addr 172.21.15.96:0/1178433741 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.513+0000 7effa237f640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c1029f0 0x7eff9c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa1b7e640 1 -- 172.21.15.96:0/1178433741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff9c108930 msgr2=0x7eff9c1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa2b80640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff9c108930 0x7eff9c1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:14.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa1b7e640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff9c108930 0x7eff9c1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa1b7e640 1 -- 172.21.15.96:0/1178433741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c1029f0 msgr2=0x7eff9c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa1b7e640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c1029f0 0x7eff9c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa1b7e640 1 -- 172.21.15.96:0/1178433741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9c1a1b90 con 0x7eff9c103c30 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa2b80640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff9c108930 0x7eff9c1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa1b7e640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c103c30 0x7eff9c19a640 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c920 tx=0x7eff9000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7eff9b7fe640 1 -- 172.21.15.96:0/1178433741 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90007c70 con 0x7eff9c103c30 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9c1a1e80 con 0x7eff9c103c30 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7eff9b7fe640 1 -- 172.21.15.96:0/1178433741 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff90007e10 con 0x7eff9c103c30 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7eff9b7fe640 1 -- 172.21.15.96:0/1178433741 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90018820 con 0x7eff9c103c30 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.514+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9c1a2340 con 0x7eff9c103c30 2024-04-25T17:08:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.515+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff9c107740 con 0x7eff9c103c30 2024-04-25T17:08:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.517+0000 7eff9b7fe640 1 -- 172.21.15.96:0/1178433741 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7eff9000f450 con 0x7eff9c103c30 2024-04-25T17:08:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.517+0000 7eff9b7fe640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7eff840afaf0 0x7eff840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.517+0000 7effa237f640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7eff840afaf0 0x7eff840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.517+0000 7eff9b7fe640 1 -- 172.21.15.96:0/1178433741 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7eff900deeb0 con 0x7eff9c103c30 2024-04-25T17:08:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.518+0000 7effa237f640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7eff840afaf0 0x7eff840b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff9c1035f0 tx=0x7eff94005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:14.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.521+0000 7eff9b7fe640 1 -- 172.21.15.96:0/1178433741 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff90014030 con 0x7eff9c103c30 2024-04-25T17:08:14.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:14 smithi171 ceph-mon[33189]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T17:08:14.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:14 smithi171 ceph-mon[33189]: Cluster is now healthy 2024-04-25T17:08:14.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:14 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/609592094' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T17:08:14.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:14 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] up:active 2024-04-25T17:08:14.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:14 smithi171 ceph-mon[33189]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:08:14.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:14 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2929647922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T17:08:14.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:14.733+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7eff9c02db50 con 0x7eff9c103c30 2024-04-25T17:08:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.390+0000 7eff9b7fe640 1 -- 172.21.15.96:0/1178433741 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v18) ==== 115+0+0 (secure 0 0 0) 0x7eff900a7c70 con 0x7eff9c103c30 2024-04-25T17:08:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7eff840afaf0 msgr2=0x7eff840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7eff840afaf0 0x7eff840b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7eff9c1035f0 tx=0x7eff94005f90 comp rx=0 tx=0).stop 2024-04-25T17:08:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c103c30 msgr2=0x7eff9c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c103c30 0x7eff9c19a640 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c920 tx=0x7eff9000cde0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 shutdown_connections 2024-04-25T17:08:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7eff840afaf0 0x7eff840b2000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff9c108930 0x7eff9c1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff9c103c30 0x7eff9c19a640 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 --2- 172.21.15.96:0/1178433741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c1029f0 0x7eff9c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 >> 172.21.15.96:0/1178433741 conn(0x7eff9c0fdbc0 msgr2=0x7eff9c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:15.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 shutdown_connections 2024-04-25T17:08:15.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.392+0000 7effa45e3640 1 -- 172.21.15.96:0/1178433741 wait complete. 2024-04-25T17:08:15.405 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T17:08:15.406 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T17:08:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 -- 172.21.15.96:0/3890692872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4103c20 msgr2=0x7f92e4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 --2- 172.21.15.96:0/3890692872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4103c20 0x7f92e4104030 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009900 tx=0x7f92d802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 -- 172.21.15.96:0/3890692872 shutdown_connections 2024-04-25T17:08:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 --2- 172.21.15.96:0/3890692872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92e4108920 0x7f92e410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 --2- 172.21.15.96:0/3890692872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4103c20 0x7f92e4104030 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 --2- 172.21.15.96:0/3890692872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e41029e0 0x7f92e41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 -- 172.21.15.96:0/3890692872 >> 172.21.15.96:0/3890692872 conn(0x7f92e40fdbd0 msgr2=0x7f92e4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 -- 172.21.15.96:0/3890692872 shutdown_connections 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.563+0000 7f92ea025640 1 -- 172.21.15.96:0/3890692872 wait complete. 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 Processor -- start 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 -- start start 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e41029e0 0x7f92e406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92e4103c20 0x7f92e419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4108920 0x7f92e41a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92e410e5e0 con 0x7f92e4108920 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92e410e460 con 0x7f92e41029e0 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92ea025640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f92e410e760 con 0x7f92e4103c20 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4108920 0x7f92e41a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4108920 0x7f92e41a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33444/0 (socket says 172.21.15.96:33444) 2024-04-25T17:08:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92e3fff640 1 -- 172.21.15.96:0/2238140326 learned_addr learned my addr 172.21.15.96:0/2238140326 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92e2ffd640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92e4103c20 0x7f92e419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.564+0000 7f92e37fe640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e41029e0 0x7f92e406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e3fff640 1 -- 172.21.15.96:0/2238140326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92e4103c20 msgr2=0x7f92e419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e3fff640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92e4103c20 0x7f92e419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e3fff640 1 -- 172.21.15.96:0/2238140326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e41029e0 msgr2=0x7f92e406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e3fff640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e41029e0 0x7f92e406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e3fff640 1 -- 172.21.15.96:0/2238140326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e41a21b0 con 0x7f92e4108920 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e37fe640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e41029e0 0x7f92e406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e3fff640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4108920 0x7f92e41a1990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00b970 tx=0x7f92dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e0ff9640 1 -- 172.21.15.96:0/2238140326 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc019070 con 0x7f92e4108920 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e0ff9640 1 -- 172.21.15.96:0/2238140326 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92dc014440 con 0x7f92e4108920 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e41a24a0 con 0x7f92e4108920 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92e0ff9640 1 -- 172.21.15.96:0/2238140326 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0075d0 con 0x7f92e4108920 2024-04-25T17:08:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.565+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e41a2a20 con 0x7f92e4108920 2024-04-25T17:08:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.566+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e4107730 con 0x7f92e4108920 2024-04-25T17:08:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.567+0000 7f92e0ff9640 1 -- 172.21.15.96:0/2238140326 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f92dc01c070 con 0x7f92e4108920 2024-04-25T17:08:15.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.568+0000 7f92e0ff9640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92cc0afaf0 0x7f92cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.568+0000 7f92e37fe640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92cc0afaf0 0x7f92cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.568+0000 7f92e0ff9640 1 -- 172.21.15.96:0/2238140326 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f92dc0de560 con 0x7f92e4108920 2024-04-25T17:08:15.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.569+0000 7f92e37fe640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92cc0afaf0 0x7f92cc0b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f92e41035e0 tx=0x7f92d4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.572+0000 7f92e0ff9640 1 -- 172.21.15.96:0/2238140326 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92dc0a6c40 con 0x7f92e4108920 2024-04-25T17:08:15.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2929647922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T17:08:15.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-mon[29131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:08:15.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-mon[29131]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:08:15.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1178433741' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T17:08:15.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-mon[29131]: pgmap v282: 97 pgs: 18 creating+peering, 12 unknown, 67 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 832 B/s wr, 3 op/s 2024-04-25T17:08:15.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:15 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2929647922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T17:08:15.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:15 smithi097 ceph-mon[32602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:08:15.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:15 smithi097 ceph-mon[32602]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:08:15.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:15 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1178433741' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T17:08:15.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:15 smithi097 ceph-mon[32602]: pgmap v282: 97 pgs: 18 creating+peering, 12 unknown, 67 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 832 B/s wr, 3 op/s 2024-04-25T17:08:15.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.778+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f92e402dd40 con 0x7f92cc0afaf0 2024-04-25T17:08:15.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:15 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2929647922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T17:08:15.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:15 smithi171 ceph-mon[33189]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:08:15.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:15 smithi171 ceph-mon[33189]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:08:15.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:15 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1178433741' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T17:08:15.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:15 smithi171 ceph-mon[33189]: pgmap v282: 97 pgs: 18 creating+peering, 12 unknown, 67 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 832 B/s wr, 3 op/s 2024-04-25T17:08:15.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.799+0000 7f92e0ff9640 1 -- 172.21.15.96:0/2238140326 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92e402dd40 con 0x7f92cc0afaf0 2024-04-25T17:08:15.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92cc0afaf0 msgr2=0x7f92cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92cc0afaf0 0x7f92cc0b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f92e41035e0 tx=0x7f92d4008040 comp rx=0 tx=0).stop 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4108920 msgr2=0x7f92e41a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4108920 0x7f92e41a1990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00b970 tx=0x7f92dc00be30 comp rx=0 tx=0).stop 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 shutdown_connections 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92e4108920 0x7f92e41a1990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92cc0afaf0 0x7f92cc0b2000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92e4103c20 0x7f92e419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 --2- 172.21.15.96:0/2238140326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92e41029e0 0x7f92e406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.801+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 >> 172.21.15.96:0/2238140326 conn(0x7f92e40fdbd0 msgr2=0x7f92e4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.802+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 shutdown_connections 2024-04-25T17:08:15.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.802+0000 7f92ea025640 1 -- 172.21.15.96:0/2238140326 wait complete. 2024-04-25T17:08:15.813 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T17:08:15.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 -- 172.21.15.96:0/1371413302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04102a00 msgr2=0x7f1a04103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 --2- 172.21.15.96:0/1371413302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04102a00 0x7f1a04103700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009900 tx=0x7f19f802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:15.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 -- 172.21.15.96:0/1371413302 shutdown_connections 2024-04-25T17:08:15.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 --2- 172.21.15.96:0/1371413302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a04108940 0x7f1a0410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 --2- 172.21.15.96:0/1371413302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a04103c40 0x7f1a04104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 --2- 172.21.15.96:0/1371413302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04102a00 0x7f1a04103700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 -- 172.21.15.96:0/1371413302 >> 172.21.15.96:0/1371413302 conn(0x7f1a040fdbf0 msgr2=0x7f1a041000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:15.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 -- 172.21.15.96:0/1371413302 shutdown_connections 2024-04-25T17:08:15.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.925+0000 7f1a090e2640 1 -- 172.21.15.96:0/1371413302 wait complete. 2024-04-25T17:08:15.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 Processor -- start 2024-04-25T17:08:15.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 -- start start 2024-04-25T17:08:15.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a04102a00 0x7f1a0419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04103c40 0x7f1a0419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a04108940 0x7f1a041a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0410f630 con 0x7f1a04103c40 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0410f4b0 con 0x7f1a04108940 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a090e2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0410f7b0 con 0x7f1a04102a00 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04103c40 0x7f1a0419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04103c40 0x7f1a0419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33480/0 (socket says 172.21.15.96:33480) 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 -- 172.21.15.96:0/782137576 learned_addr learned my addr 172.21.15.96:0/782137576 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a02d9d640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a04102a00 0x7f1a0419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0359e640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a04108940 0x7f1a041a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 -- 172.21.15.96:0/782137576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a04102a00 msgr2=0x7f1a0419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a04102a00 0x7f1a0419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 -- 172.21.15.96:0/782137576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a04108940 msgr2=0x7f1a041a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a04108940 0x7f1a041a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.926+0000 7f1a0259c640 1 -- 172.21.15.96:0/782137576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a041a1d70 con 0x7f1a04103c40 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.927+0000 7f1a0259c640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04103c40 0x7f1a0419a7e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f19f400d950 tx=0x7f19f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.927+0000 7f19ebfff640 1 -- 172.21.15.96:0/782137576 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4017070 con 0x7f1a04103c40 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.927+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a041a2060 con 0x7f1a04103c40 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.927+0000 7f19ebfff640 1 -- 172.21.15.96:0/782137576 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19f4009c30 con 0x7f1a04103c40 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.927+0000 7f19ebfff640 1 -- 172.21.15.96:0/782137576 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f400f620 con 0x7f1a04103c40 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.927+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a041a25b0 con 0x7f1a04103c40 2024-04-25T17:08:15.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.928+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a04107750 con 0x7f1a04103c40 2024-04-25T17:08:15.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.929+0000 7f19ebfff640 1 -- 172.21.15.96:0/782137576 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f19f4010460 con 0x7f1a04103c40 2024-04-25T17:08:15.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.930+0000 7f19ebfff640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f19ec0afaf0 0x7f19ec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:15.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.930+0000 7f1a02d9d640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f19ec0afaf0 0x7f19ec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:15.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.930+0000 7f19ebfff640 1 -- 172.21.15.96:0/782137576 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f19f40de760 con 0x7f1a04103c40 2024-04-25T17:08:15.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.930+0000 7f1a02d9d640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f19ec0afaf0 0x7f19ec0b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009230 tx=0x7f19f8031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:15.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:15.933+0000 7f19ebfff640 1 -- 172.21.15.96:0/782137576 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19f40a7490 con 0x7f1a04103c40 2024-04-25T17:08:16.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.137+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1a04063070 con 0x7f19ec0afaf0 2024-04-25T17:08:16.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.162+0000 7f19ebfff640 1 -- 172.21.15.96:0/782137576 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1a04063070 con 0x7f19ec0afaf0 2024-04-25T17:08:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f19ec0afaf0 msgr2=0x7f19ec0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f19ec0afaf0 0x7f19ec0b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009230 tx=0x7f19f8031400 comp rx=0 tx=0).stop 2024-04-25T17:08:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04103c40 msgr2=0x7f1a0419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04103c40 0x7f1a0419a7e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f19f400d950 tx=0x7f19f400de10 comp rx=0 tx=0).stop 2024-04-25T17:08:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 shutdown_connections 2024-04-25T17:08:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a04108940 0x7f1a041a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a04103c40 0x7f1a0419a7e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f19ec0afaf0 0x7f19ec0b2000 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 --2- 172.21.15.96:0/782137576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a04102a00 0x7f1a0419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 >> 172.21.15.96:0/782137576 conn(0x7f1a040fdbf0 msgr2=0x7f1a04104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:16.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 shutdown_connections 2024-04-25T17:08:16.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.164+0000 7f1a090e2640 1 -- 172.21.15.96:0/782137576 wait complete. 2024-04-25T17:08:16.174 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:08:15.783+0000 7ff852aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:08:16.174 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:08:15.783+0000 7ff852aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:08:16.174 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:08:15.783+0000 7ff852aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:08:16.174 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:08:15.783+0000 7ff852aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:08:16.174 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:08:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:08:15.783+0000 7ff852aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:08:16.176 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T17:08:16.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 -- 172.21.15.96:0/757077041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29481045d0 msgr2=0x7f29481069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 --2- 172.21.15.96:0/757077041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29481045d0 0x7f29481069f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f293802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:16.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 -- 172.21.15.96:0/757077041 shutdown_connections 2024-04-25T17:08:16.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 --2- 172.21.15.96:0/757077041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948109890 0x7f294810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 --2- 172.21.15.96:0/757077041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2948106f30 0x7f2948109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 --2- 172.21.15.96:0/757077041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29481045d0 0x7f29481069f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 -- 172.21.15.96:0/757077041 >> 172.21.15.96:0/757077041 conn(0x7f29480fdbd0 msgr2=0x7f2948100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.289+0000 7f294d620640 1 -- 172.21.15.96:0/757077041 shutdown_connections 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 -- 172.21.15.96:0/757077041 wait complete. 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 Processor -- start 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 -- start start 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29481045d0 0x7f294819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948106f30 0x7f294819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2948109890 0x7f29481a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f294810f5c0 con 0x7f2948109890 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f294810f440 con 0x7f29481045d0 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f294d620640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f294810f740 con 0x7f2948106f30 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f29477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2948109890 0x7f29481a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f29477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2948109890 0x7f29481a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33492/0 (socket says 172.21.15.96:33492) 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.290+0000 7f29477fe640 1 -- 172.21.15.96:0/1960207775 learned_addr learned my addr 172.21.15.96:0/1960207775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:16.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f2946ffd640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29481045d0 0x7f294819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f293ffff640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948106f30 0x7f294819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f29477fe640 1 -- 172.21.15.96:0/1960207775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948106f30 msgr2=0x7f294819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f29477fe640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948106f30 0x7f294819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f29477fe640 1 -- 172.21.15.96:0/1960207775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29481045d0 msgr2=0x7f294819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f29477fe640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29481045d0 0x7f294819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f29477fe640 1 -- 172.21.15.96:0/1960207775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29481a20e0 con 0x7f2948109890 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f29477fe640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2948109890 0x7f29481a1a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f294000cad0 tx=0x7f294000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f2944ff9640 1 -- 172.21.15.96:0/1960207775 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294000ae20 con 0x7f2948109890 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f2944ff9640 1 -- 172.21.15.96:0/1960207775 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2940003e40 con 0x7f2948109890 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29481a23d0 con 0x7f2948109890 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f2944ff9640 1 -- 172.21.15.96:0/1960207775 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294000f6b0 con 0x7f2948109890 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.291+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29481a28d0 con 0x7f2948109890 2024-04-25T17:08:16.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.292+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294806c2a0 con 0x7f2948109890 2024-04-25T17:08:16.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.293+0000 7f2944ff9640 1 -- 172.21.15.96:0/1960207775 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f29400042c0 con 0x7f2948109890 2024-04-25T17:08:16.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.294+0000 7f2944ff9640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f29340afaf0 0x7f29340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.294+0000 7f2946ffd640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f29340afaf0 0x7f29340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.294+0000 7f2944ff9640 1 -- 172.21.15.96:0/1960207775 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f29400deef0 con 0x7f2948109890 2024-04-25T17:08:16.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.294+0000 7f2946ffd640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f29340afaf0 0x7f29340b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2938009230 tx=0x7f2938031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:16.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.298+0000 7f2944ff9640 1 -- 172.21.15.96:0/1960207775 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2940014070 con 0x7f2948109890 2024-04-25T17:08:16.391 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:08:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b[62836]: 2024-04-25T17:08:16.111+0000 7f76f5acf700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T17:08:16.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:16 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1178433741' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T17:08:16.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:16 smithi096 ceph-mon[29131]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:08:16.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:16 smithi096 ceph-mon[29131]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:16.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.506+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2948105370 con 0x7f29340afaf0 2024-04-25T17:08:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.511+0000 7f2944ff9640 1 -- 172.21.15.96:0/1960207775 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2948105370 con 0x7f29340afaf0 2024-04-25T17:08:16.511 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_0/75fd11cc-e253-401c-982e-bf2a9ad2810f 2024-04-25T17:08:16.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f29340afaf0 msgr2=0x7f29340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f29340afaf0 0x7f29340b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2938009230 tx=0x7f2938031400 comp rx=0 tx=0).stop 2024-04-25T17:08:16.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2948109890 msgr2=0x7f29481a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2948109890 0x7f29481a1a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f294000cad0 tx=0x7f294000cf90 comp rx=0 tx=0).stop 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 shutdown_connections 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2948109890 0x7f29481a1a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f29340afaf0 0x7f29340b2000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948106f30 0x7f294819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 --2- 172.21.15.96:0/1960207775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29481045d0 0x7f294819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 >> 172.21.15.96:0/1960207775 conn(0x7f29480fdbd0 msgr2=0x7f2948104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 shutdown_connections 2024-04-25T17:08:16.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.513+0000 7f294d620640 1 -- 172.21.15.96:0/1960207775 wait complete. 2024-04-25T17:08:16.524 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 -- 172.21.15.96:0/2148585781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103af0 msgr2=0x7fc34c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 --2- 172.21.15.96:0/2148585781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103af0 0x7fc34c1047f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc348009900 tx=0x7fc34802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 -- 172.21.15.96:0/2148585781 shutdown_connections 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 --2- 172.21.15.96:0/2148585781 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc34c109a30 0x7fc34c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 --2- 172.21.15.96:0/2148585781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc34c104d30 0x7fc34c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 --2- 172.21.15.96:0/2148585781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103af0 0x7fc34c1047f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 -- 172.21.15.96:0/2148585781 >> 172.21.15.96:0/2148585781 conn(0x7fc34c0fece0 msgr2=0x7fc34c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 -- 172.21.15.96:0/2148585781 shutdown_connections 2024-04-25T17:08:16.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 -- 172.21.15.96:0/2148585781 wait complete. 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 Processor -- start 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.636+0000 7fc353946640 1 -- start start 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc34c103af0 0x7fc34c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc34c104d30 0x7fc34c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c109a30 0x7fc34c1a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c10f670 con 0x7fc34c109a30 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c10f4f0 con 0x7fc34c103af0 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c10f7f0 con 0x7fc34c104d30 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c109a30 0x7fc34c1a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c109a30 0x7fc34c1a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33512/0 (socket says 172.21.15.96:33512) 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 -- 172.21.15.96:0/337299596 learned_addr learned my addr 172.21.15.96:0/337299596 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:16.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc350ee1640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc34c104d30 0x7fc34c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc3516e2640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc34c103af0 0x7fc34c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 -- 172.21.15.96:0/337299596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc34c104d30 msgr2=0x7fc34c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc34c104d30 0x7fc34c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 -- 172.21.15.96:0/337299596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc34c103af0 msgr2=0x7fc34c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc34c103af0 0x7fc34c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 -- 172.21.15.96:0/337299596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c1a1de0 con 0x7fc34c109a30 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc3516e2640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc34c103af0 0x7fc34c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc351ee3640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c109a30 0x7fc34c1a1770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc34400eaa0 tx=0x7fc34400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc3427fc640 1 -- 172.21.15.96:0/337299596 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344004d20 con 0x7fc34c109a30 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc3427fc640 1 -- 172.21.15.96:0/337299596 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc344005020 con 0x7fc34c109a30 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c1a20d0 con 0x7fc34c109a30 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc3427fc640 1 -- 172.21.15.96:0/337299596 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3440116b0 con 0x7fc34c109a30 2024-04-25T17:08:16.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.637+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c1a26a0 con 0x7fc34c109a30 2024-04-25T17:08:16.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.638+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c108840 con 0x7fc34c109a30 2024-04-25T17:08:16.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.640+0000 7fc3427fc640 1 -- 172.21.15.96:0/337299596 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc3440051d0 con 0x7fc34c109a30 2024-04-25T17:08:16.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.640+0000 7fc3427fc640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc3380afaf0 0x7fc3380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:16.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.640+0000 7fc3516e2640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc3380afaf0 0x7fc3380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:16.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.640+0000 7fc3427fc640 1 -- 172.21.15.96:0/337299596 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fc3440ded00 con 0x7fc34c109a30 2024-04-25T17:08:16.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.641+0000 7fc3516e2640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc3380afaf0 0x7fc3380b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc348009230 tx=0x7fc348000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:16.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.645+0000 7fc3427fc640 1 -- 172.21.15.96:0/337299596 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc344016030 con 0x7fc34c109a30 2024-04-25T17:08:16.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:16 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1178433741' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T17:08:16.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:16 smithi097 ceph-mon[32602]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:08:16.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:16 smithi097 ceph-mon[32602]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:16.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:16 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1178433741' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T17:08:16.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:16 smithi171 ceph-mon[33189]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:08:16.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:16 smithi171 ceph-mon[33189]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:16.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.847+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc34c062fe0 con 0x7fc3380afaf0 2024-04-25T17:08:16.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.874+0000 7fc3427fc640 1 -- 172.21.15.96:0/337299596 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc34c062fe0 con 0x7fc3380afaf0 2024-04-25T17:08:16.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc3380afaf0 msgr2=0x7fc3380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc3380afaf0 0x7fc3380b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc348009230 tx=0x7fc348000bc0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c109a30 msgr2=0x7fc34c1a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c109a30 0x7fc34c1a1770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc34400eaa0 tx=0x7fc34400eea0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 shutdown_connections 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c109a30 0x7fc34c1a1770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc3380afaf0 0x7fc3380b2000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc34c104d30 0x7fc34c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.876+0000 7fc353946640 1 --2- 172.21.15.96:0/337299596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc34c103af0 0x7fc34c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.877+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 >> 172.21.15.96:0/337299596 conn(0x7fc34c0fece0 msgr2=0x7fc34c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.877+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 shutdown_connections 2024-04-25T17:08:16.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:16.877+0000 7fc353946640 1 -- 172.21.15.96:0/337299596 wait complete. 2024-04-25T17:08:16.887 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 -- 172.21.15.96:0/3934278364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4101e30 msgr2=0x7fa4d4102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/3934278364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4101e30 0x7fa4d4102240 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009900 tx=0x7fa4c402a410 comp rx=0 tx=0).stop 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 -- 172.21.15.96:0/3934278364 shutdown_connections 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/3934278364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4109780 0x7fa4d410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/3934278364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4106c10 0x7fa4d41090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/3934278364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4101e30 0x7fa4d4102240 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 -- 172.21.15.96:0/3934278364 >> 172.21.15.96:0/3934278364 conn(0x7fa4d40fd340 msgr2=0x7fa4d40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 -- 172.21.15.96:0/3934278364 shutdown_connections 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 -- 172.21.15.96:0/3934278364 wait complete. 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 Processor -- start 2024-04-25T17:08:17.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.000+0000 7fa4d9fee640 1 -- start start 2024-04-25T17:08:17.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d9fee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4101e30 0x7fa4d419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d9fee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4106c10 0x7fa4d419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d9fee640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4109780 0x7fa4d41a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d9fee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410f5e0 con 0x7fa4d4106c10 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d9fee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410f460 con 0x7fa4d4101e30 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d9fee640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410f760 con 0x7fa4d4109780 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4106c10 0x7fa4d419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4106c10 0x7fa4d419a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33530/0 (socket says 172.21.15.96:33530) 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 -- 172.21.15.96:0/1076079105 learned_addr learned my addr 172.21.15.96:0/1076079105 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d37fe640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4101e30 0x7fa4d419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d3fff640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4109780 0x7fa4d41a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 -- 172.21.15.96:0/1076079105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4109780 msgr2=0x7fa4d41a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4109780 0x7fa4d41a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 -- 172.21.15.96:0/1076079105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4101e30 msgr2=0x7fa4d419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4101e30 0x7fa4d419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 -- 172.21.15.96:0/1076079105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d41a1d40 con 0x7fa4d4106c10 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d2ffd640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4106c10 0x7fa4d419a790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800c970 tx=0x7fa4c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d0ff9640 1 -- 172.21.15.96:0/1076079105 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8007bb0 con 0x7fa4d4106c10 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d0ff9640 1 -- 172.21.15.96:0/1076079105 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4c8007d50 con 0x7fa4d4106c10 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d41a1fd0 con 0x7fa4d4106c10 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.001+0000 7fa4d0ff9640 1 -- 172.21.15.96:0/1076079105 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c800f450 con 0x7fa4d4106c10 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.002+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41a2490 con 0x7fa4d4106c10 2024-04-25T17:08:17.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.003+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d40fee60 con 0x7fa4d4106c10 2024-04-25T17:08:17.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.005+0000 7fa4d0ff9640 1 -- 172.21.15.96:0/1076079105 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa4c8004f00 con 0x7fa4d4106c10 2024-04-25T17:08:17.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.005+0000 7fa4d0ff9640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4bc0afaf0 0x7fa4bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.005+0000 7fa4d37fe640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4bc0afaf0 0x7fa4bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.005+0000 7fa4d0ff9640 1 -- 172.21.15.96:0/1076079105 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fa4c80decc0 con 0x7fa4d4106c10 2024-04-25T17:08:17.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.006+0000 7fa4d37fe640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4bc0afaf0 0x7fa4bc0b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4000c00 tx=0x7fa4c4031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:17.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.009+0000 7fa4d0ff9640 1 -- 172.21.15.96:0/1076079105 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4c8014030 con 0x7fa4d4106c10 2024-04-25T17:08:17.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.217+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa4d4062fe0 con 0x7fa4bc0afaf0 2024-04-25T17:08:17.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.223+0000 7fa4d0ff9640 1 -- 172.21.15.96:0/1076079105 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa4d4062fe0 con 0x7fa4bc0afaf0 2024-04-25T17:08:17.222 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_1/87e8df53-ac6d-498d-95ee-67a03515551e 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4bc0afaf0 msgr2=0x7fa4bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4bc0afaf0 0x7fa4bc0b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4000c00 tx=0x7fa4c4031400 comp rx=0 tx=0).stop 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4106c10 msgr2=0x7fa4d419a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4106c10 0x7fa4d419a790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800c970 tx=0x7fa4c800ce30 comp rx=0 tx=0).stop 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 shutdown_connections 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fa4bc0afaf0 0x7fa4bc0b2000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d4109780 0x7fa4d41a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4d4106c10 0x7fa4d419a790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 --2- 172.21.15.96:0/1076079105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4d4101e30 0x7fa4d419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 >> 172.21.15.96:0/1076079105 conn(0x7fa4d40fd340 msgr2=0x7fa4d4108d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:17.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 shutdown_connections 2024-04-25T17:08:17.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.225+0000 7fa4d9fee640 1 -- 172.21.15.96:0/1076079105 wait complete. 2024-04-25T17:08:17.236 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 -- 172.21.15.96:0/3219195391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f0102180 msgr2=0x7f28f0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/3219195391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f0102180 0x7f28f0102e80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009900 tx=0x7f28e402a410 comp rx=0 tx=0).stop 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 -- 172.21.15.96:0/3219195391 shutdown_connections 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/3219195391 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28f01080c0 0x7f28f010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/3219195391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f01033c0 0x7f28f01037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/3219195391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f0102180 0x7f28f0102e80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 -- 172.21.15.96:0/3219195391 >> 172.21.15.96:0/3219195391 conn(0x7f28f00fd370 msgr2=0x7f28f00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 -- 172.21.15.96:0/3219195391 shutdown_connections 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 -- 172.21.15.96:0/3219195391 wait complete. 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 Processor -- start 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.343+0000 7f28f6b2d640 1 -- start start 2024-04-25T17:08:17.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0102180 0x7f28f010a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28f01033c0 0x7f28f019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f01080c0 0x7f28f01a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010dca0 con 0x7f28f01080c0 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010db20 con 0x7f28f0102180 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f28f010de20 con 0x7f28f01033c0 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f01080c0 0x7f28f01a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f01080c0 0x7f28f01a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33554/0 (socket says 172.21.15.96:33554) 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 -- 172.21.15.96:0/2344384353 learned_addr learned my addr 172.21.15.96:0/2344384353 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28effff640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28f01033c0 0x7f28f019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f48c9640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0102180 0x7f28f010a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 -- 172.21.15.96:0/2344384353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28f01033c0 msgr2=0x7f28f019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28f01033c0 0x7f28f019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 -- 172.21.15.96:0/2344384353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0102180 msgr2=0x7f28f010a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0102180 0x7f28f010a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 -- 172.21.15.96:0/2344384353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f01a1d30 con 0x7f28f01080c0 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f48c9640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0102180 0x7f28f010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f50ca640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f01080c0 0x7f28f01a1510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28e800cad0 tx=0x7f28e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800ae20 con 0x7f28f01080c0 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e8005020 con 0x7f28f01080c0 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f01a1fc0 con 0x7f28f01080c0 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800f6b0 con 0x7f28f01080c0 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.344+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a24d0 con 0x7f28f01080c0 2024-04-25T17:08:17.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.345+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f01077c0 con 0x7f28f01080c0 2024-04-25T17:08:17.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.347+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f28e80051d0 con 0x7f28f01080c0 2024-04-25T17:08:17.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.347+0000 7f28edffb640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28e00afb70 0x7f28e00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.347+0000 7f28f48c9640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28e00afb70 0x7f28e00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.347+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f28e80deb10 con 0x7f28f01080c0 2024-04-25T17:08:17.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.348+0000 7f28f48c9640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28e00afb70 0x7f28e00b2080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f28e4005f70 tx=0x7f28e4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:17.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.352+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28e8014030 con 0x7f28f01080c0 2024-04-25T17:08:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.408+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f28e80a33d0 con 0x7f28f01080c0 2024-04-25T17:08:17.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.575+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28f002db50 con 0x7f28f01080c0 2024-04-25T17:08:17.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.576+0000 7f28edffb640 1 -- 172.21.15.96:0/2344384353 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11805 (secure 0 0 0) 0x7f28e80d15c0 con 0x7f28f01080c0 2024-04-25T17:08:17.575 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:17.576 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:08:14.735284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:08:17.576 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28e00afb70 msgr2=0x7f28e00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28e00afb70 0x7f28e00b2080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f28e4005f70 tx=0x7f28e4000bc0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f01080c0 msgr2=0x7f28f01a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f01080c0 0x7f28f01a1510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28e800cad0 tx=0x7f28e800cf90 comp rx=0 tx=0).stop 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 shutdown_connections 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28f01080c0 0x7f28f01a1510 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f28e00afb70 0x7f28e00b2080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28f01033c0 0x7f28f019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 --2- 172.21.15.96:0/2344384353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28f0102180 0x7f28f010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 >> 172.21.15.96:0/2344384353 conn(0x7f28f00fd370 msgr2=0x7f28f00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:17.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 shutdown_connections 2024-04-25T17:08:17.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.578+0000 7f28f6b2d640 1 -- 172.21.15.96:0/2344384353 wait complete. 2024-04-25T17:08:17.588 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:17 smithi096 ceph-mon[29131]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.588 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:17 smithi096 ceph-mon[29131]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.588 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:17 smithi096 ceph-mon[29131]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.588 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:17 smithi096 ceph-mon[29131]: pgmap v283: 97 pgs: 5 creating+peering, 92 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T17:08:17.589 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:08:17.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 -- 172.21.15.96:0/2251358199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 msgr2=0x7f3758109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 --2- 172.21.15.96:0/2251358199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f3758109330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3748009ca0 tx=0x7f374802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 -- 172.21.15.96:0/2251358199 shutdown_connections 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 --2- 172.21.15.96:0/2251358199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3758109870 0x7f375810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 --2- 172.21.15.96:0/2251358199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f3758109330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 --2- 172.21.15.96:0/2251358199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37581045b0 0x7f37581069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 -- 172.21.15.96:0/2251358199 >> 172.21.15.96:0/2251358199 conn(0x7f37580fdbd0 msgr2=0x7f3758100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 -- 172.21.15.96:0/2251358199 shutdown_connections 2024-04-25T17:08:17.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 -- 172.21.15.96:0/2251358199 wait complete. 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 Processor -- start 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 -- start start 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.696+0000 7f375ede3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37581045b0 0x7f3758103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375ede3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f37581020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375ede3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3758109870 0x7f37581025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375ede3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f375810f5d0 con 0x7f3758106f10 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375ede3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f375810f450 con 0x7f3758109870 2024-04-25T17:08:17.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375ede3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f375810f750 con 0x7f37581045b0 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f37581020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f37581020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47616/0 (socket says 172.21.15.96:47616) 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 -- 172.21.15.96:0/3819931771 learned_addr learned my addr 172.21.15.96:0/3819931771 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375d380640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3758109870 0x7f37581025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375cb7f640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37581045b0 0x7f3758103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 -- 172.21.15.96:0/3819931771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37581045b0 msgr2=0x7f3758103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37581045b0 0x7f3758103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 -- 172.21.15.96:0/3819931771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3758109870 msgr2=0x7f37581025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3758109870 0x7f37581025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 -- 172.21.15.96:0/3819931771 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3758102d10 con 0x7f3758106f10 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3757fff640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f37581020a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f374802ade0 tx=0x7f3748003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3755ffb640 1 -- 172.21.15.96:0/3819931771 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374802f380 con 0x7f3758106f10 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37581a42f0 con 0x7f3758106f10 2024-04-25T17:08:17.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3755ffb640 1 -- 172.21.15.96:0/3819931771 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37480317c0 con 0x7f3758106f10 2024-04-25T17:08:17.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.697+0000 7f3755ffb640 1 -- 172.21.15.96:0/3819931771 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374802fc60 con 0x7f3758106f10 2024-04-25T17:08:17.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.698+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37581a4750 con 0x7f3758106f10 2024-04-25T17:08:17.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.699+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375806afa0 con 0x7f3758106f10 2024-04-25T17:08:17.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.700+0000 7f3755ffb640 1 -- 172.21.15.96:0/3819931771 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f3748004040 con 0x7f3758106f10 2024-04-25T17:08:17.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.700+0000 7f3755ffb640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f37440afd20 0x7f37440b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:17.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.701+0000 7f375cb7f640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f37440afd20 0x7f37440b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:17.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.701+0000 7f3755ffb640 1 -- 172.21.15.96:0/3819931771 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f3748100190 con 0x7f3758106f10 2024-04-25T17:08:17.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.701+0000 7f375cb7f640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f37440afd20 0x7f37440b2230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f375806a8b0 tx=0x7f3740005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:17.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.704+0000 7f3755ffb640 1 -- 172.21.15.96:0/3819931771 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37480c8ed0 con 0x7f3758106f10 2024-04-25T17:08:17.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:17 smithi097 ceph-mon[32602]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:17 smithi097 ceph-mon[32602]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:17 smithi097 ceph-mon[32602]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:17 smithi097 ceph-mon[32602]: pgmap v283: 97 pgs: 5 creating+peering, 92 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T17:08:17.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:17 smithi171 ceph-mon[33189]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:17 smithi171 ceph-mon[33189]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:17 smithi171 ceph-mon[33189]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:17.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:17 smithi171 ceph-mon[33189]: pgmap v283: 97 pgs: 5 creating+peering, 92 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T17:08:17.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.881+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37580637a0 con 0x7f3758106f10 2024-04-25T17:08:17.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.883+0000 7f3755ffb640 1 -- 172.21.15.96:0/3819931771 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22171 (secure 0 0 0) 0x7f37480c87d0 con 0x7f3758106f10 2024-04-25T17:08:17.883 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:17.883 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":76,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:08:13.137147+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f37440afd20 msgr2=0x7f37440b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f37440afd20 0x7f37440b2230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f375806a8b0 tx=0x7f3740005f90 comp rx=0 tx=0).stop 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 msgr2=0x7f37581020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f37581020a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f374802ade0 tx=0x7f3748003ba0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 shutdown_connections 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3758109870 0x7f37581025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3758106f10 0x7f37581020a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f37440afd20 0x7f37440b2230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 --2- 172.21.15.96:0/3819931771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37581045b0 0x7f3758103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:17.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 >> 172.21.15.96:0/3819931771 conn(0x7f37580fdbd0 msgr2=0x7f3758105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:17.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 shutdown_connections 2024-04-25T17:08:17.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:17.885+0000 7f375ede3640 1 -- 172.21.15.96:0/3819931771 wait complete. 2024-04-25T17:08:17.897 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T17:08:17.897 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T17:08:18.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 -- 172.21.15.96:0/1132187334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 msgr2=0x7f2a841054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 --2- 172.21.15.96:0/1132187334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841054b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009900 tx=0x7f2a7802a410 comp rx=0 tx=0).stop 2024-04-25T17:08:18.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 -- 172.21.15.96:0/1132187334 shutdown_connections 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 --2- 172.21.15.96:0/1132187334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a84102250 0x7f2a8410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 --2- 172.21.15.96:0/1132187334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a84101750 0x7f2a84101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 --2- 172.21.15.96:0/1132187334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841054b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 -- 172.21.15.96:0/1132187334 >> 172.21.15.96:0/1132187334 conn(0x7f2a840fd370 msgr2=0x7f2a840ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 -- 172.21.15.96:0/1132187334 shutdown_connections 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.005+0000 7f2a8b651640 1 -- 172.21.15.96:0/1132187334 wait complete. 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 Processor -- start 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 -- start start 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a84101750 0x7f2a8419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a84102250 0x7f2a8419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410f610 con 0x7f2a841050a0 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410f490 con 0x7f2a84101750 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410f790 con 0x7f2a84102250 2024-04-25T17:08:18.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47630/0 (socket says 172.21.15.96:47630) 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 -- 172.21.15.96:0/2949190388 learned_addr learned my addr 172.21.15.96:0/2949190388 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 -- 172.21.15.96:0/2949190388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a84102250 msgr2=0x7f2a8419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a84102250 0x7f2a8419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 -- 172.21.15.96:0/2949190388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a84101750 msgr2=0x7f2a8419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a84101750 0x7f2a8419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 -- 172.21.15.96:0/2949190388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a841a1d60 con 0x7f2a841050a0 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a89bee640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841a16f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00eaa0 tx=0x7f2a7c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a767fc640 1 -- 172.21.15.96:0/2949190388 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00cdb0 con 0x7f2a841050a0 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a841a1ff0 con 0x7f2a841050a0 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a767fc640 1 -- 172.21.15.96:0/2949190388 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c005020 con 0x7f2a841050a0 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a841a2560 con 0x7f2a841050a0 2024-04-25T17:08:18.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.006+0000 7f2a767fc640 1 -- 172.21.15.96:0/2949190388 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0116d0 con 0x7f2a841050a0 2024-04-25T17:08:18.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.007+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a84108a30 con 0x7f2a841050a0 2024-04-25T17:08:18.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.009+0000 7f2a767fc640 1 -- 172.21.15.96:0/2949190388 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2a7c0054a0 con 0x7f2a841050a0 2024-04-25T17:08:18.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.010+0000 7f2a767fc640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2a700afb80 0x7f2a700b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.010+0000 7f2a893ed640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2a700afb80 0x7f2a700b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.010+0000 7f2a767fc640 1 -- 172.21.15.96:0/2949190388 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f2a7c0df220 con 0x7f2a841050a0 2024-04-25T17:08:18.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.010+0000 7f2a893ed640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2a700afb80 0x7f2a700b2090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a78031500 tx=0x7f2a78031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:18.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.014+0000 7f2a767fc640 1 -- 172.21.15.96:0/2949190388 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a7c016030 con 0x7f2a841050a0 2024-04-25T17:08:18.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.191+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a84062fe0 con 0x7f2a700afb80 2024-04-25T17:08:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.198+0000 7f2a767fc640 1 -- 172.21.15.96:0/2949190388 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258464 (secure 0 0 0) 0x7f2a84062fe0 con 0x7f2a700afb80 2024-04-25T17:08:18.201 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:18.203 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2a700afb80 msgr2=0x7f2a700b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2a700afb80 0x7f2a700b2090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a78031500 tx=0x7f2a78031400 comp rx=0 tx=0).stop 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 msgr2=0x7f2a841a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841a16f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00eaa0 tx=0x7f2a7c00ef60 comp rx=0 tx=0).stop 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 shutdown_connections 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a841050a0 0x7f2a841a16f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2a700afb80 0x7f2a700b2090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2a84102250 0x7f2a8419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 --2- 172.21.15.96:0/2949190388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a84101750 0x7f2a8419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 >> 172.21.15.96:0/2949190388 conn(0x7f2a840fd370 msgr2=0x7f2a84105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:18.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 shutdown_connections 2024-04-25T17:08:18.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.201+0000 7f2a8b651640 1 -- 172.21.15.96:0/2949190388 wait complete. 2024-04-25T17:08:18.212 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-04-25T17:08:17.049972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":85,"num_read_kb":73,"num_write":162,"num_write_kb":2162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358832,"kb_used_data":9620,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582992,"statfs":{"total":1151940427776,"available":1151572983808,"internally_reserved":0,"allocated":9850880,"data_stored":6379550,"data_compressed":107190,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001410"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167139+0000","last_change":"2024-04-25T17:08:12.162430+0000","last_active":"2024-04-25T17:08:12.167139+0000","last_peered":"2024-04-25T17:08:12.167139+0000","last_clean":"2024-04-25T17:08:12.167139+0000","last_became_active":"2024-04-25T17:08:12.162052+0000","last_became_peered":"2024-04-25T17:08:12.162052+0000","last_unstale":"2024-04-25T17:08:12.167139+0000","last_undegraded":"2024-04-25T17:08:12.167139+0000","last_fullsized":"2024-04-25T17:08:12.167139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:14.457325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421957+0000","last_change":"2024-04-25T17:08:12.169139+0000","last_active":"2024-04-25T17:08:13.421957+0000","last_peered":"2024-04-25T17:08:13.421957+0000","last_clean":"2024-04-25T17:08:13.421957+0000","last_became_active":"2024-04-25T17:08:12.168715+0000","last_became_peered":"2024-04-25T17:08:12.168715+0000","last_unstale":"2024-04-25T17:08:13.421957+0000","last_undegraded":"2024-04-25T17:08:13.421957+0000","last_fullsized":"2024-04-25T17:08:13.421957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:57.357750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421733+0000","last_change":"2024-04-25T17:08:12.159141+0000","last_active":"2024-04-25T17:08:13.421733+0000","last_peered":"2024-04-25T17:08:13.421733+0000","last_clean":"2024-04-25T17:08:13.421733+0000","last_became_active":"2024-04-25T17:08:12.158796+0000","last_became_peered":"2024-04-25T17:08:12.158796+0000","last_unstale":"2024-04-25T17:08:13.421733+0000","last_undegraded":"2024-04-25T17:08:13.421733+0000","last_fullsized":"2024-04-25T17:08:13.421733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:55.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.161521+0000","last_change":"2024-04-25T17:08:12.160464+0000","last_active":"2024-04-25T17:08:12.161521+0000","last_peered":"2024-04-25T17:08:12.161521+0000","last_clean":"2024-04-25T17:08:12.161521+0000","last_became_active":"2024-04-25T17:08:12.160118+0000","last_became_peered":"2024-04-25T17:08:12.160118+0000","last_unstale":"2024-04-25T17:08:12.161521+0000","last_undegraded":"2024-04-25T17:08:12.161521+0000","last_fullsized":"2024-04-25T17:08:12.161521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:57.579423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495515+0000","last_change":"2024-04-25T17:08:12.174591+0000","last_active":"2024-04-25T17:08:13.495515+0000","last_peered":"2024-04-25T17:08:13.495515+0000","last_clean":"2024-04-25T17:08:13.495515+0000","last_became_active":"2024-04-25T17:08:12.174173+0000","last_became_peered":"2024-04-25T17:08:12.174173+0000","last_unstale":"2024-04-25T17:08:13.495515+0000","last_undegraded":"2024-04-25T17:08:13.495515+0000","last_fullsized":"2024-04-25T17:08:13.495515+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:12.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141095+0000","last_change":"2024-04-25T17:08:12.164230+0000","last_active":"2024-04-25T17:08:13.141095+0000","last_peered":"2024-04-25T17:08:13.141095+0000","last_clean":"2024-04-25T17:08:13.141095+0000","last_became_active":"2024-04-25T17:08:12.163868+0000","last_became_peered":"2024-04-25T17:08:12.163868+0000","last_unstale":"2024-04-25T17:08:13.141095+0000","last_undegraded":"2024-04-25T17:08:13.141095+0000","last_fullsized":"2024-04-25T17:08:13.141095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:12.604164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121611+0000","last_change":"2024-04-25T17:08:12.161627+0000","last_active":"2024-04-25T17:08:14.121611+0000","last_peered":"2024-04-25T17:08:14.121611+0000","last_clean":"2024-04-25T17:08:14.121611+0000","last_became_active":"2024-04-25T17:08:12.161146+0000","last_became_peered":"2024-04-25T17:08:12.161146+0000","last_unstale":"2024-04-25T17:08:14.121611+0000","last_undegraded":"2024-04-25T17:08:14.121611+0000","last_fullsized":"2024-04-25T17:08:14.121611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.972053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.166908+0000","last_change":"2024-04-25T17:08:12.165512+0000","last_active":"2024-04-25T17:08:12.166908+0000","last_peered":"2024-04-25T17:08:12.166908+0000","last_clean":"2024-04-25T17:08:12.166908+0000","last_became_active":"2024-04-25T17:08:12.164018+0000","last_became_peered":"2024-04-25T17:08:12.164018+0000","last_unstale":"2024-04-25T17:08:12.166908+0000","last_undegraded":"2024-04-25T17:08:12.166908+0000","last_fullsized":"2024-04-25T17:08:12.166908+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:45:29.650067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495771+0000","last_change":"2024-04-25T17:08:12.173313+0000","last_active":"2024-04-25T17:08:13.495771+0000","last_peered":"2024-04-25T17:08:13.495771+0000","last_clean":"2024-04-25T17:08:13.495771+0000","last_became_active":"2024-04-25T17:08:12.171269+0000","last_became_peered":"2024-04-25T17:08:12.171269+0000","last_unstale":"2024-04-25T17:08:13.495771+0000","last_undegraded":"2024-04-25T17:08:13.495771+0000","last_fullsized":"2024-04-25T17:08:13.495771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:39.565973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141559+0000","last_change":"2024-04-25T17:08:12.166114+0000","last_active":"2024-04-25T17:08:13.141559+0000","last_peered":"2024-04-25T17:08:13.141559+0000","last_clean":"2024-04-25T17:08:13.141559+0000","last_became_active":"2024-04-25T17:08:12.165737+0000","last_became_peered":"2024-04-25T17:08:12.165737+0000","last_unstale":"2024-04-25T17:08:13.141559+0000","last_undegraded":"2024-04-25T17:08:13.141559+0000","last_fullsized":"2024-04-25T17:08:13.141559+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:55.624239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122116+0000","last_change":"2024-04-25T17:08:12.160435+0000","last_active":"2024-04-25T17:08:14.122116+0000","last_peered":"2024-04-25T17:08:14.122116+0000","last_clean":"2024-04-25T17:08:14.122116+0000","last_became_active":"2024-04-25T17:08:12.160085+0000","last_became_peered":"2024-04-25T17:08:12.160085+0000","last_unstale":"2024-04-25T17:08:14.122116+0000","last_undegraded":"2024-04-25T17:08:14.122116+0000","last_fullsized":"2024-04-25T17:08:14.122116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.709567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174761+0000","last_change":"2024-04-25T17:08:12.173990+0000","last_active":"2024-04-25T17:08:12.174761+0000","last_peered":"2024-04-25T17:08:12.174761+0000","last_clean":"2024-04-25T17:08:12.174761+0000","last_became_active":"2024-04-25T17:08:12.172838+0000","last_became_peered":"2024-04-25T17:08:12.172838+0000","last_unstale":"2024-04-25T17:08:12.174761+0000","last_undegraded":"2024-04-25T17:08:12.174761+0000","last_fullsized":"2024-04-25T17:08:12.174761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:16.784335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.169084+0000","last_change":"2024-04-25T17:08:12.167579+0000","last_active":"2024-04-25T17:08:12.169084+0000","last_peered":"2024-04-25T17:08:12.169084+0000","last_clean":"2024-04-25T17:08:12.169084+0000","last_became_active":"2024-04-25T17:08:12.166583+0000","last_became_peered":"2024-04-25T17:08:12.166583+0000","last_unstale":"2024-04-25T17:08:12.169084+0000","last_undegraded":"2024-04-25T17:08:12.169084+0000","last_fullsized":"2024-04-25T17:08:12.169084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:55.855450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421592+0000","last_change":"2024-04-25T17:08:12.161263+0000","last_active":"2024-04-25T17:08:13.421592+0000","last_peered":"2024-04-25T17:08:13.421592+0000","last_clean":"2024-04-25T17:08:13.421592+0000","last_became_active":"2024-04-25T17:08:12.160078+0000","last_became_peered":"2024-04-25T17:08:12.160078+0000","last_unstale":"2024-04-25T17:08:13.421592+0000","last_undegraded":"2024-04-25T17:08:13.421592+0000","last_fullsized":"2024-04-25T17:08:13.421592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:56.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421594+0000","last_change":"2024-04-25T17:08:12.167318+0000","last_active":"2024-04-25T17:08:13.421594+0000","last_peered":"2024-04-25T17:08:13.421594+0000","last_clean":"2024-04-25T17:08:13.421594+0000","last_became_active":"2024-04-25T17:08:12.166964+0000","last_became_peered":"2024-04-25T17:08:12.166964+0000","last_unstale":"2024-04-25T17:08:13.421594+0000","last_undegraded":"2024-04-25T17:08:13.421594+0000","last_fullsized":"2024-04-25T17:08:13.421594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:25.967818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494787+0000","last_change":"2024-04-25T17:08:12.166417+0000","last_active":"2024-04-25T17:08:13.494787+0000","last_peered":"2024-04-25T17:08:13.494787+0000","last_clean":"2024-04-25T17:08:13.494787+0000","last_became_active":"2024-04-25T17:08:12.165928+0000","last_became_peered":"2024-04-25T17:08:12.165928+0000","last_unstale":"2024-04-25T17:08:13.494787+0000","last_undegraded":"2024-04-25T17:08:13.494787+0000","last_fullsized":"2024-04-25T17:08:13.494787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:24:22.359814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167674+0000","last_change":"2024-04-25T17:08:12.164952+0000","last_active":"2024-04-25T17:08:12.167674+0000","last_peered":"2024-04-25T17:08:12.167674+0000","last_clean":"2024-04-25T17:08:12.167674+0000","last_became_active":"2024-04-25T17:08:12.164595+0000","last_became_peered":"2024-04-25T17:08:12.164595+0000","last_unstale":"2024-04-25T17:08:12.167674+0000","last_undegraded":"2024-04-25T17:08:12.167674+0000","last_fullsized":"2024-04-25T17:08:12.167674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:48.456115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.154737+0000","last_change":"2024-04-25T17:08:12.153583+0000","last_active":"2024-04-25T17:08:12.154737+0000","last_peered":"2024-04-25T17:08:12.154737+0000","last_clean":"2024-04-25T17:08:12.154737+0000","last_became_active":"2024-04-25T17:08:12.153070+0000","last_became_peered":"2024-04-25T17:08:12.153070+0000","last_unstale":"2024-04-25T17:08:12.154737+0000","last_undegraded":"2024-04-25T17:08:12.154737+0000","last_fullsized":"2024-04-25T17:08:12.154737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352707+0000","last_change":"2024-04-25T17:08:12.166837+0000","last_active":"2024-04-25T17:08:13.352707+0000","last_peered":"2024-04-25T17:08:13.352707+0000","last_clean":"2024-04-25T17:08:13.352707+0000","last_became_active":"2024-04-25T17:08:12.166477+0000","last_became_peered":"2024-04-25T17:08:12.166477+0000","last_unstale":"2024-04-25T17:08:13.352707+0000","last_undegraded":"2024-04-25T17:08:13.352707+0000","last_fullsized":"2024-04-25T17:08:13.352707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:50.492333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.170659+0000","last_change":"2024-04-25T17:08:11.153164+0000","last_active":"2024-04-25T17:08:12.170659+0000","last_peered":"2024-04-25T17:08:12.170659+0000","last_clean":"2024-04-25T17:08:12.170659+0000","last_became_active":"2024-04-25T17:08:11.152189+0000","last_became_peered":"2024-04-25T17:08:11.152189+0000","last_unstale":"2024-04-25T17:08:12.170659+0000","last_undegraded":"2024-04-25T17:08:12.170659+0000","last_fullsized":"2024-04-25T17:08:12.170659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:30.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.178622+0000","last_change":"2024-04-25T17:08:12.178117+0000","last_active":"2024-04-25T17:08:12.178622+0000","last_peered":"2024-04-25T17:08:12.178622+0000","last_clean":"2024-04-25T17:08:12.178622+0000","last_became_active":"2024-04-25T17:08:12.177767+0000","last_became_peered":"2024-04-25T17:08:12.177767+0000","last_unstale":"2024-04-25T17:08:12.178622+0000","last_undegraded":"2024-04-25T17:08:12.178622+0000","last_fullsized":"2024-04-25T17:08:12.178622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:41.464565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.179579+0000","last_change":"2024-04-25T17:08:11.153996+0000","last_active":"2024-04-25T17:08:12.179579+0000","last_peered":"2024-04-25T17:08:12.179579+0000","last_clean":"2024-04-25T17:08:12.179579+0000","last_became_active":"2024-04-25T17:08:11.152993+0000","last_became_peered":"2024-04-25T17:08:11.152993+0000","last_unstale":"2024-04-25T17:08:12.179579+0000","last_undegraded":"2024-04-25T17:08:12.179579+0000","last_fullsized":"2024-04-25T17:08:12.179579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:33.116616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422141+0000","last_change":"2024-04-25T17:08:12.163707+0000","last_active":"2024-04-25T17:08:13.422141+0000","last_peered":"2024-04-25T17:08:13.422141+0000","last_clean":"2024-04-25T17:08:13.422141+0000","last_became_active":"2024-04-25T17:08:12.161868+0000","last_became_peered":"2024-04-25T17:08:12.161868+0000","last_unstale":"2024-04-25T17:08:13.422141+0000","last_undegraded":"2024-04-25T17:08:13.422141+0000","last_fullsized":"2024-04-25T17:08:13.422141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.333036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496262+0000","last_change":"2024-04-25T17:08:11.158101+0000","last_active":"2024-04-25T17:08:13.496262+0000","last_peered":"2024-04-25T17:08:13.496262+0000","last_clean":"2024-04-25T17:08:13.496262+0000","last_became_active":"2024-04-25T17:08:11.157747+0000","last_became_peered":"2024-04-25T17:08:11.157747+0000","last_unstale":"2024-04-25T17:08:13.496262+0000","last_undegraded":"2024-04-25T17:08:13.496262+0000","last_fullsized":"2024-04-25T17:08:13.496262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:03.840823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495644+0000","last_change":"2024-04-25T17:08:12.170278+0000","last_active":"2024-04-25T17:08:13.495644+0000","last_peered":"2024-04-25T17:08:13.495644+0000","last_clean":"2024-04-25T17:08:13.495644+0000","last_became_active":"2024-04-25T17:08:12.169219+0000","last_became_peered":"2024-04-25T17:08:12.169219+0000","last_unstale":"2024-04-25T17:08:13.495644+0000","last_undegraded":"2024-04-25T17:08:13.495644+0000","last_fullsized":"2024-04-25T17:08:13.495644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:35.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496183+0000","last_change":"2024-04-25T17:08:11.153167+0000","last_active":"2024-04-25T17:08:13.496183+0000","last_peered":"2024-04-25T17:08:13.496183+0000","last_clean":"2024-04-25T17:08:13.496183+0000","last_became_active":"2024-04-25T17:08:11.152772+0000","last_became_peered":"2024-04-25T17:08:11.152772+0000","last_unstale":"2024-04-25T17:08:13.496183+0000","last_undegraded":"2024-04-25T17:08:13.496183+0000","last_fullsized":"2024-04-25T17:08:13.496183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:57.198963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.146203+0000","last_change":"2024-04-25T17:08:11.146203+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.146203+0000","last_undegraded":"2024-04-25T17:08:11.146203+0000","last_fullsized":"2024-04-25T17:08:11.146203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.159382+0000","last_change":"2024-04-25T17:08:11.158564+0000","last_active":"2024-04-25T17:08:11.159382+0000","last_peered":"2024-04-25T17:08:11.159382+0000","last_clean":"2024-04-25T17:08:11.159382+0000","last_became_active":"2024-04-25T17:08:11.158143+0000","last_became_peered":"2024-04-25T17:08:11.158143+0000","last_unstale":"2024-04-25T17:08:11.159382+0000","last_undegraded":"2024-04-25T17:08:11.159382+0000","last_fullsized":"2024-04-25T17:08:11.159382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:44.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495878+0000","last_change":"2024-04-25T17:08:12.167584+0000","last_active":"2024-04-25T17:08:13.495878+0000","last_peered":"2024-04-25T17:08:13.495878+0000","last_clean":"2024-04-25T17:08:13.495878+0000","last_became_active":"2024-04-25T17:08:12.166717+0000","last_became_peered":"2024-04-25T17:08:12.166717+0000","last_unstale":"2024-04-25T17:08:13.495878+0000","last_undegraded":"2024-04-25T17:08:13.495878+0000","last_fullsized":"2024-04-25T17:08:13.495878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:38.047671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141009+0000","last_change":"2024-04-25T17:08:11.159254+0000","last_active":"2024-04-25T17:08:13.141009+0000","last_peered":"2024-04-25T17:08:13.141009+0000","last_clean":"2024-04-25T17:08:13.141009+0000","last_became_active":"2024-04-25T17:08:11.158864+0000","last_became_peered":"2024-04-25T17:08:11.158864+0000","last_unstale":"2024-04-25T17:08:13.141009+0000","last_undegraded":"2024-04-25T17:08:13.141009+0000","last_fullsized":"2024-04-25T17:08:13.141009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:17.795010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.144854+0000","last_change":"2024-04-25T17:08:11.144854+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.144854+0000","last_undegraded":"2024-04-25T17:08:11.144854+0000","last_fullsized":"2024-04-25T17:08:11.144854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.148562+0000","last_change":"2024-04-25T17:08:11.150980+0000","last_active":"2024-04-25T17:08:12.148562+0000","last_peered":"2024-04-25T17:08:12.148562+0000","last_clean":"2024-04-25T17:08:12.148562+0000","last_became_active":"2024-04-25T17:08:11.150580+0000","last_became_peered":"2024-04-25T17:08:11.150580+0000","last_unstale":"2024-04-25T17:08:12.148562+0000","last_undegraded":"2024-04-25T17:08:12.148562+0000","last_fullsized":"2024-04-25T17:08:12.148562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:59.539673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.175719+0000","last_change":"2024-04-25T17:08:12.171013+0000","last_active":"2024-04-25T17:08:12.175719+0000","last_peered":"2024-04-25T17:08:12.175719+0000","last_clean":"2024-04-25T17:08:12.175719+0000","last_became_active":"2024-04-25T17:08:12.169868+0000","last_became_peered":"2024-04-25T17:08:12.169868+0000","last_unstale":"2024-04-25T17:08:12.175719+0000","last_undegraded":"2024-04-25T17:08:12.175719+0000","last_fullsized":"2024-04-25T17:08:12.175719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:59.044063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.158357+0000","last_change":"2024-04-25T17:08:11.157618+0000","last_active":"2024-04-25T17:08:11.158357+0000","last_peered":"2024-04-25T17:08:11.158357+0000","last_clean":"2024-04-25T17:08:11.158357+0000","last_became_active":"2024-04-25T17:08:11.157052+0000","last_became_peered":"2024-04-25T17:08:11.157052+0000","last_unstale":"2024-04-25T17:08:11.158357+0000","last_undegraded":"2024-04-25T17:08:11.158357+0000","last_fullsized":"2024-04-25T17:08:11.158357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:09.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.151617+0000","last_change":"2024-04-25T17:08:12.151093+0000","last_active":"2024-04-25T17:08:12.151617+0000","last_peered":"2024-04-25T17:08:12.151617+0000","last_clean":"2024-04-25T17:08:12.151617+0000","last_became_active":"2024-04-25T17:08:12.150699+0000","last_became_peered":"2024-04-25T17:08:12.150699+0000","last_unstale":"2024-04-25T17:08:12.151617+0000","last_undegraded":"2024-04-25T17:08:12.151617+0000","last_fullsized":"2024-04-25T17:08:12.151617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:46.331493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141389+0000","last_change":"2024-04-25T17:08:11.157673+0000","last_active":"2024-04-25T17:08:13.141389+0000","last_peered":"2024-04-25T17:08:13.141389+0000","last_clean":"2024-04-25T17:08:13.141389+0000","last_became_active":"2024-04-25T17:08:11.157089+0000","last_became_peered":"2024-04-25T17:08:11.157089+0000","last_unstale":"2024-04-25T17:08:13.141389+0000","last_undegraded":"2024-04-25T17:08:13.141389+0000","last_fullsized":"2024-04-25T17:08:13.141389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:47.586003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495779+0000","last_change":"2024-04-25T17:08:12.170666+0000","last_active":"2024-04-25T17:08:13.495779+0000","last_peered":"2024-04-25T17:08:13.495779+0000","last_clean":"2024-04-25T17:08:13.495779+0000","last_became_active":"2024-04-25T17:08:12.169607+0000","last_became_peered":"2024-04-25T17:08:12.169607+0000","last_unstale":"2024-04-25T17:08:13.495779+0000","last_undegraded":"2024-04-25T17:08:13.495779+0000","last_fullsized":"2024-04-25T17:08:13.495779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121887+0000","last_change":"2024-04-25T17:08:11.153407+0000","last_active":"2024-04-25T17:08:14.121887+0000","last_peered":"2024-04-25T17:08:14.121887+0000","last_clean":"2024-04-25T17:08:14.121887+0000","last_became_active":"2024-04-25T17:08:11.152932+0000","last_became_peered":"2024-04-25T17:08:11.152932+0000","last_unstale":"2024-04-25T17:08:14.121887+0000","last_undegraded":"2024-04-25T17:08:14.121887+0000","last_fullsized":"2024-04-25T17:08:14.121887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:06.712620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422086+0000","last_change":"2024-04-25T17:08:12.173788+0000","last_active":"2024-04-25T17:08:13.422086+0000","last_peered":"2024-04-25T17:08:13.422086+0000","last_clean":"2024-04-25T17:08:13.422086+0000","last_became_active":"2024-04-25T17:08:12.172714+0000","last_became_peered":"2024-04-25T17:08:12.172714+0000","last_unstale":"2024-04-25T17:08:13.422086+0000","last_undegraded":"2024-04-25T17:08:13.422086+0000","last_fullsized":"2024-04-25T17:08:13.422086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:18.892187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.161140+0000","last_change":"2024-04-25T17:08:11.159802+0000","last_active":"2024-04-25T17:08:11.161140+0000","last_peered":"2024-04-25T17:08:11.161140+0000","last_clean":"2024-04-25T17:08:11.161140+0000","last_became_active":"2024-04-25T17:08:11.159431+0000","last_became_peered":"2024-04-25T17:08:11.159431+0000","last_unstale":"2024-04-25T17:08:11.161140+0000","last_undegraded":"2024-04-25T17:08:11.161140+0000","last_fullsized":"2024-04-25T17:08:11.161140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:45.961985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495822+0000","last_change":"2024-04-25T17:08:12.162703+0000","last_active":"2024-04-25T17:08:13.495822+0000","last_peered":"2024-04-25T17:08:13.495822+0000","last_clean":"2024-04-25T17:08:13.495822+0000","last_became_active":"2024-04-25T17:08:12.162361+0000","last_became_peered":"2024-04-25T17:08:12.162361+0000","last_unstale":"2024-04-25T17:08:13.495822+0000","last_undegraded":"2024-04-25T17:08:13.495822+0000","last_fullsized":"2024-04-25T17:08:13.495822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:51.262319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'6","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141260+0000","last_change":"2024-04-25T17:08:11.153938+0000","last_active":"2024-04-25T17:08:13.141260+0000","last_peered":"2024-04-25T17:08:13.141260+0000","last_clean":"2024-04-25T17:08:13.141260+0000","last_became_active":"2024-04-25T17:08:11.153591+0000","last_became_peered":"2024-04-25T17:08:11.153591+0000","last_unstale":"2024-04-25T17:08:13.141260+0000","last_undegraded":"2024-04-25T17:08:13.141260+0000","last_fullsized":"2024-04-25T17:08:13.141260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496204+0000","last_change":"2024-04-25T17:08:12.156045+0000","last_active":"2024-04-25T17:08:13.496204+0000","last_peered":"2024-04-25T17:08:13.496204+0000","last_clean":"2024-04-25T17:08:13.496204+0000","last_became_active":"2024-04-25T17:08:12.155657+0000","last_became_peered":"2024-04-25T17:08:12.155657+0000","last_unstale":"2024-04-25T17:08:13.496204+0000","last_undegraded":"2024-04-25T17:08:13.496204+0000","last_fullsized":"2024-04-25T17:08:13.496204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352496+0000","last_change":"2024-04-25T17:08:11.162009+0000","last_active":"2024-04-25T17:08:13.352496+0000","last_peered":"2024-04-25T17:08:13.352496+0000","last_clean":"2024-04-25T17:08:13.352496+0000","last_became_active":"2024-04-25T17:08:11.161574+0000","last_became_peered":"2024-04-25T17:08:11.161574+0000","last_unstale":"2024-04-25T17:08:13.352496+0000","last_undegraded":"2024-04-25T17:08:13.352496+0000","last_fullsized":"2024-04-25T17:08:13.352496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:16.760355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163159+0000","last_change":"2024-04-25T17:08:12.159522+0000","last_active":"2024-04-25T17:08:12.163159+0000","last_peered":"2024-04-25T17:08:12.163159+0000","last_clean":"2024-04-25T17:08:12.163159+0000","last_became_active":"2024-04-25T17:08:12.159036+0000","last_became_peered":"2024-04-25T17:08:12.159036+0000","last_unstale":"2024-04-25T17:08:12.163159+0000","last_undegraded":"2024-04-25T17:08:12.163159+0000","last_fullsized":"2024-04-25T17:08:12.163159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:18.534810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.162996+0000","last_change":"2024-04-25T17:08:11.162487+0000","last_active":"2024-04-25T17:08:11.162996+0000","last_peered":"2024-04-25T17:08:11.162996+0000","last_clean":"2024-04-25T17:08:11.162996+0000","last_became_active":"2024-04-25T17:08:11.161961+0000","last_became_peered":"2024-04-25T17:08:11.161961+0000","last_unstale":"2024-04-25T17:08:11.162996+0000","last_undegraded":"2024-04-25T17:08:11.162996+0000","last_fullsized":"2024-04-25T17:08:11.162996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:09.095387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121943+0000","last_change":"2024-04-25T17:08:12.164107+0000","last_active":"2024-04-25T17:08:14.121943+0000","last_peered":"2024-04-25T17:08:14.121943+0000","last_clean":"2024-04-25T17:08:14.121943+0000","last_became_active":"2024-04-25T17:08:12.163741+0000","last_became_peered":"2024-04-25T17:08:12.163741+0000","last_unstale":"2024-04-25T17:08:14.121943+0000","last_undegraded":"2024-04-25T17:08:14.121943+0000","last_fullsized":"2024-04-25T17:08:14.121943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.552833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.146579+0000","last_change":"2024-04-25T17:08:11.153383+0000","last_active":"2024-04-25T17:08:12.146579+0000","last_peered":"2024-04-25T17:08:12.146579+0000","last_clean":"2024-04-25T17:08:12.146579+0000","last_became_active":"2024-04-25T17:08:11.152573+0000","last_became_peered":"2024-04-25T17:08:11.152573+0000","last_unstale":"2024-04-25T17:08:12.146579+0000","last_undegraded":"2024-04-25T17:08:12.146579+0000","last_fullsized":"2024-04-25T17:08:12.146579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:13.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121535+0000","last_change":"2024-04-25T17:08:12.165939+0000","last_active":"2024-04-25T17:08:14.121535+0000","last_peered":"2024-04-25T17:08:14.121535+0000","last_clean":"2024-04-25T17:08:14.121535+0000","last_became_active":"2024-04-25T17:08:12.165569+0000","last_became_peered":"2024-04-25T17:08:12.165569+0000","last_unstale":"2024-04-25T17:08:14.121535+0000","last_undegraded":"2024-04-25T17:08:14.121535+0000","last_fullsized":"2024-04-25T17:08:14.121535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:15:16.230457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495142+0000","last_change":"2024-04-25T17:08:11.158927+0000","last_active":"2024-04-25T17:08:13.495142+0000","last_peered":"2024-04-25T17:08:13.495142+0000","last_clean":"2024-04-25T17:08:13.495142+0000","last_became_active":"2024-04-25T17:08:11.158528+0000","last_became_peered":"2024-04-25T17:08:11.158528+0000","last_unstale":"2024-04-25T17:08:13.495142+0000","last_undegraded":"2024-04-25T17:08:13.495142+0000","last_fullsized":"2024-04-25T17:08:13.495142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.308627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.143338+0000","last_change":"2024-04-25T17:08:11.143338+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.143338+0000","last_undegraded":"2024-04-25T17:08:11.143338+0000","last_fullsized":"2024-04-25T17:08:11.143338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495648+0000","last_change":"2024-04-25T17:08:11.153214+0000","last_active":"2024-04-25T17:08:13.495648+0000","last_peered":"2024-04-25T17:08:13.495648+0000","last_clean":"2024-04-25T17:08:13.495648+0000","last_became_active":"2024-04-25T17:08:11.152728+0000","last_became_peered":"2024-04-25T17:08:11.152728+0000","last_unstale":"2024-04-25T17:08:13.495648+0000","last_undegraded":"2024-04-25T17:08:13.495648+0000","last_fullsized":"2024-04-25T17:08:13.495648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.338549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167104+0000","last_change":"2024-04-25T17:08:12.162167+0000","last_active":"2024-04-25T17:08:12.167104+0000","last_peered":"2024-04-25T17:08:12.167104+0000","last_clean":"2024-04-25T17:08:12.167104+0000","last_became_active":"2024-04-25T17:08:12.160896+0000","last_became_peered":"2024-04-25T17:08:12.160896+0000","last_unstale":"2024-04-25T17:08:12.167104+0000","last_undegraded":"2024-04-25T17:08:12.167104+0000","last_fullsized":"2024-04-25T17:08:12.167104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:20.437234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.141261+0000","last_change":"2024-04-25T17:08:11.158995+0000","last_active":"2024-04-25T17:08:12.141261+0000","last_peered":"2024-04-25T17:08:12.141261+0000","last_clean":"2024-04-25T17:08:12.141261+0000","last_became_active":"2024-04-25T17:08:11.158644+0000","last_became_peered":"2024-04-25T17:08:11.158644+0000","last_unstale":"2024-04-25T17:08:12.141261+0000","last_undegraded":"2024-04-25T17:08:12.141261+0000","last_fullsized":"2024-04-25T17:08:12.141261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:22.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141392+0000","last_change":"2024-04-25T17:08:12.161635+0000","last_active":"2024-04-25T17:08:13.141392+0000","last_peered":"2024-04-25T17:08:13.141392+0000","last_clean":"2024-04-25T17:08:13.141392+0000","last_became_active":"2024-04-25T17:08:12.153728+0000","last_became_peered":"2024-04-25T17:08:12.153728+0000","last_unstale":"2024-04-25T17:08:13.141392+0000","last_undegraded":"2024-04-25T17:08:13.141392+0000","last_fullsized":"2024-04-25T17:08:13.141392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:02.003550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121630+0000","last_change":"2024-04-25T17:08:11.150490+0000","last_active":"2024-04-25T17:08:14.121630+0000","last_peered":"2024-04-25T17:08:14.121630+0000","last_clean":"2024-04-25T17:08:14.121630+0000","last_became_active":"2024-04-25T17:08:11.148391+0000","last_became_peered":"2024-04-25T17:08:11.148391+0000","last_unstale":"2024-04-25T17:08:14.121630+0000","last_undegraded":"2024-04-25T17:08:14.121630+0000","last_fullsized":"2024-04-25T17:08:14.121630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:40.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141295+0000","last_change":"2024-04-25T17:08:12.168136+0000","last_active":"2024-04-25T17:08:13.141295+0000","last_peered":"2024-04-25T17:08:13.141295+0000","last_clean":"2024-04-25T17:08:13.141295+0000","last_became_active":"2024-04-25T17:08:12.167786+0000","last_became_peered":"2024-04-25T17:08:12.167786+0000","last_unstale":"2024-04-25T17:08:13.141295+0000","last_undegraded":"2024-04-25T17:08:13.141295+0000","last_fullsized":"2024-04-25T17:08:13.141295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:22.869503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.149194+0000","last_change":"2024-04-25T17:08:11.149194+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.149194+0000","last_undegraded":"2024-04-25T17:08:11.149194+0000","last_fullsized":"2024-04-25T17:08:11.149194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.149557+0000","last_change":"2024-04-25T17:08:11.149085+0000","last_active":"2024-04-25T17:08:12.149557+0000","last_peered":"2024-04-25T17:08:12.149557+0000","last_clean":"2024-04-25T17:08:12.149557+0000","last_became_active":"2024-04-25T17:08:11.148712+0000","last_became_peered":"2024-04-25T17:08:11.148712+0000","last_unstale":"2024-04-25T17:08:12.149557+0000","last_undegraded":"2024-04-25T17:08:12.149557+0000","last_fullsized":"2024-04-25T17:08:12.149557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:12.764034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.178626+0000","last_change":"2024-04-25T17:08:12.178154+0000","last_active":"2024-04-25T17:08:12.178626+0000","last_peered":"2024-04-25T17:08:12.178626+0000","last_clean":"2024-04-25T17:08:12.178626+0000","last_became_active":"2024-04-25T17:08:12.177773+0000","last_became_peered":"2024-04-25T17:08:12.177773+0000","last_unstale":"2024-04-25T17:08:12.178626+0000","last_undegraded":"2024-04-25T17:08:12.178626+0000","last_fullsized":"2024-04-25T17:08:12.178626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:30.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.148011+0000","last_change":"2024-04-25T17:08:11.152932+0000","last_active":"2024-04-25T17:08:12.148011+0000","last_peered":"2024-04-25T17:08:12.148011+0000","last_clean":"2024-04-25T17:08:12.148011+0000","last_became_active":"2024-04-25T17:08:11.152530+0000","last_became_peered":"2024-04-25T17:08:11.152530+0000","last_unstale":"2024-04-25T17:08:12.148011+0000","last_undegraded":"2024-04-25T17:08:12.148011+0000","last_fullsized":"2024-04-25T17:08:12.148011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121425+0000","last_change":"2024-04-25T17:06:55.951326+0000","last_active":"2024-04-25T17:08:14.121425+0000","last_peered":"2024-04-25T17:08:14.121425+0000","last_clean":"2024-04-25T17:08:14.121425+0000","last_became_active":"2024-04-25T17:06:55.946641+0000","last_became_peered":"2024-04-25T17:06:55.946641+0000","last_unstale":"2024-04-25T17:08:14.121425+0000","last_undegraded":"2024-04-25T17:08:14.121425+0000","last_fullsized":"2024-04-25T17:08:14.121425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_clean_scrub_stamp":"2024-04-25T17:03:27.394276+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:58.308843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163189+0000","last_change":"2024-04-25T17:08:12.162065+0000","last_active":"2024-04-25T17:08:12.163189+0000","last_peered":"2024-04-25T17:08:12.163189+0000","last_clean":"2024-04-25T17:08:12.163189+0000","last_became_active":"2024-04-25T17:08:12.161733+0000","last_became_peered":"2024-04-25T17:08:12.161733+0000","last_unstale":"2024-04-25T17:08:12.163189+0000","last_undegraded":"2024-04-25T17:08:12.163189+0000","last_fullsized":"2024-04-25T17:08:12.163189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:05.132252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121612+0000","last_change":"2024-04-25T17:08:11.155941+0000","last_active":"2024-04-25T17:08:14.121612+0000","last_peered":"2024-04-25T17:08:14.121612+0000","last_clean":"2024-04-25T17:08:14.121612+0000","last_became_active":"2024-04-25T17:08:11.155381+0000","last_became_peered":"2024-04-25T17:08:11.155381+0000","last_unstale":"2024-04-25T17:08:14.121612+0000","last_undegraded":"2024-04-25T17:08:14.121612+0000","last_fullsized":"2024-04-25T17:08:14.121612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:28.423462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496103+0000","last_change":"2024-04-25T17:08:11.158960+0000","last_active":"2024-04-25T17:08:13.496103+0000","last_peered":"2024-04-25T17:08:13.496103+0000","last_clean":"2024-04-25T17:08:13.496103+0000","last_became_active":"2024-04-25T17:08:11.158609+0000","last_became_peered":"2024-04-25T17:08:11.158609+0000","last_unstale":"2024-04-25T17:08:13.496103+0000","last_undegraded":"2024-04-25T17:08:13.496103+0000","last_fullsized":"2024-04-25T17:08:13.496103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:19.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163020+0000","last_change":"2024-04-25T17:08:12.161483+0000","last_active":"2024-04-25T17:08:12.163020+0000","last_peered":"2024-04-25T17:08:12.163020+0000","last_clean":"2024-04-25T17:08:12.163020+0000","last_became_active":"2024-04-25T17:08:12.158586+0000","last_became_peered":"2024-04-25T17:08:12.158586+0000","last_unstale":"2024-04-25T17:08:12.163020+0000","last_undegraded":"2024-04-25T17:08:12.163020+0000","last_fullsized":"2024-04-25T17:08:12.163020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.141542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.141400+0000","last_change":"2024-04-25T17:08:11.161497+0000","last_active":"2024-04-25T17:08:12.141400+0000","last_peered":"2024-04-25T17:08:12.141400+0000","last_clean":"2024-04-25T17:08:12.141400+0000","last_became_active":"2024-04-25T17:08:11.160990+0000","last_became_peered":"2024-04-25T17:08:11.160990+0000","last_unstale":"2024-04-25T17:08:12.141400+0000","last_undegraded":"2024-04-25T17:08:12.141400+0000","last_fullsized":"2024-04-25T17:08:12.141400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:59.682133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176351+0000","last_change":"2024-04-25T17:08:12.171214+0000","last_active":"2024-04-25T17:08:12.176351+0000","last_peered":"2024-04-25T17:08:12.176351+0000","last_clean":"2024-04-25T17:08:12.176351+0000","last_became_active":"2024-04-25T17:08:12.170877+0000","last_became_peered":"2024-04-25T17:08:12.170877+0000","last_unstale":"2024-04-25T17:08:12.176351+0000","last_undegraded":"2024-04-25T17:08:12.176351+0000","last_fullsized":"2024-04-25T17:08:12.176351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:18.267340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176348+0000","last_change":"2024-04-25T17:08:11.154819+0000","last_active":"2024-04-25T17:08:12.176348+0000","last_peered":"2024-04-25T17:08:12.176348+0000","last_clean":"2024-04-25T17:08:12.176348+0000","last_became_active":"2024-04-25T17:08:11.154006+0000","last_became_peered":"2024-04-25T17:08:11.154006+0000","last_unstale":"2024-04-25T17:08:12.176348+0000","last_undegraded":"2024-04-25T17:08:12.176348+0000","last_fullsized":"2024-04-25T17:08:12.176348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.063058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422056+0000","last_change":"2024-04-25T17:08:12.167301+0000","last_active":"2024-04-25T17:08:13.422056+0000","last_peered":"2024-04-25T17:08:13.422056+0000","last_clean":"2024-04-25T17:08:13.422056+0000","last_became_active":"2024-04-25T17:08:12.166944+0000","last_became_peered":"2024-04-25T17:08:12.166944+0000","last_unstale":"2024-04-25T17:08:13.422056+0000","last_undegraded":"2024-04-25T17:08:13.422056+0000","last_fullsized":"2024-04-25T17:08:13.422056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:46.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495108+0000","last_change":"2024-04-25T17:08:11.156616+0000","last_active":"2024-04-25T17:08:13.495108+0000","last_peered":"2024-04-25T17:08:13.495108+0000","last_clean":"2024-04-25T17:08:13.495108+0000","last_became_active":"2024-04-25T17:08:11.156256+0000","last_became_peered":"2024-04-25T17:08:11.156256+0000","last_unstale":"2024-04-25T17:08:13.495108+0000","last_undegraded":"2024-04-25T17:08:13.495108+0000","last_fullsized":"2024-04-25T17:08:13.495108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:37.539773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495319+0000","last_change":"2024-04-25T17:08:12.175216+0000","last_active":"2024-04-25T17:08:13.495319+0000","last_peered":"2024-04-25T17:08:13.495319+0000","last_clean":"2024-04-25T17:08:13.495319+0000","last_became_active":"2024-04-25T17:08:12.174521+0000","last_became_peered":"2024-04-25T17:08:12.174521+0000","last_unstale":"2024-04-25T17:08:13.495319+0000","last_undegraded":"2024-04-25T17:08:13.495319+0000","last_fullsized":"2024-04-25T17:08:13.495319+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.322733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.152221+0000","last_change":"2024-04-25T17:08:11.162287+0000","last_active":"2024-04-25T17:08:12.152221+0000","last_peered":"2024-04-25T17:08:12.152221+0000","last_clean":"2024-04-25T17:08:12.152221+0000","last_became_active":"2024-04-25T17:08:11.158297+0000","last_became_peered":"2024-04-25T17:08:11.158297+0000","last_unstale":"2024-04-25T17:08:12.152221+0000","last_undegraded":"2024-04-25T17:08:12.152221+0000","last_fullsized":"2024-04-25T17:08:12.152221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:15.758880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167929+0000","last_change":"2024-04-25T17:08:12.163535+0000","last_active":"2024-04-25T17:08:12.167929+0000","last_peered":"2024-04-25T17:08:12.167929+0000","last_clean":"2024-04-25T17:08:12.167929+0000","last_became_active":"2024-04-25T17:08:12.162848+0000","last_became_peered":"2024-04-25T17:08:12.162848+0000","last_unstale":"2024-04-25T17:08:12.167929+0000","last_undegraded":"2024-04-25T17:08:12.167929+0000","last_fullsized":"2024-04-25T17:08:12.167929+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:36.839617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352187+0000","last_change":"2024-04-25T17:08:11.162780+0000","last_active":"2024-04-25T17:08:13.352187+0000","last_peered":"2024-04-25T17:08:13.352187+0000","last_clean":"2024-04-25T17:08:13.352187+0000","last_became_active":"2024-04-25T17:08:11.162385+0000","last_became_peered":"2024-04-25T17:08:11.162385+0000","last_unstale":"2024-04-25T17:08:13.352187+0000","last_undegraded":"2024-04-25T17:08:13.352187+0000","last_fullsized":"2024-04-25T17:08:13.352187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:25.420740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174051+0000","last_change":"2024-04-25T17:08:12.170828+0000","last_active":"2024-04-25T17:08:12.174051+0000","last_peered":"2024-04-25T17:08:12.174051+0000","last_clean":"2024-04-25T17:08:12.174051+0000","last_became_active":"2024-04-25T17:08:12.170479+0000","last_became_peered":"2024-04-25T17:08:12.170479+0000","last_unstale":"2024-04-25T17:08:12.174051+0000","last_undegraded":"2024-04-25T17:08:12.174051+0000","last_fullsized":"2024-04-25T17:08:12.174051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:30.418656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495662+0000","last_change":"2024-04-25T17:08:11.156754+0000","last_active":"2024-04-25T17:08:13.495662+0000","last_peered":"2024-04-25T17:08:13.495662+0000","last_clean":"2024-04-25T17:08:13.495662+0000","last_became_active":"2024-04-25T17:08:11.156414+0000","last_became_peered":"2024-04-25T17:08:11.156414+0000","last_unstale":"2024-04-25T17:08:13.495662+0000","last_undegraded":"2024-04-25T17:08:13.495662+0000","last_fullsized":"2024-04-25T17:08:13.495662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:10.364234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121978+0000","last_change":"2024-04-25T17:08:12.162032+0000","last_active":"2024-04-25T17:08:14.121978+0000","last_peered":"2024-04-25T17:08:14.121978+0000","last_clean":"2024-04-25T17:08:14.121978+0000","last_became_active":"2024-04-25T17:08:12.161017+0000","last_became_peered":"2024-04-25T17:08:12.161017+0000","last_unstale":"2024-04-25T17:08:14.121978+0000","last_undegraded":"2024-04-25T17:08:14.121978+0000","last_fullsized":"2024-04-25T17:08:14.121978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.472984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352660+0000","last_change":"2024-04-25T17:08:11.149550+0000","last_active":"2024-04-25T17:08:13.352660+0000","last_peered":"2024-04-25T17:08:13.352660+0000","last_clean":"2024-04-25T17:08:13.352660+0000","last_became_active":"2024-04-25T17:08:11.149082+0000","last_became_peered":"2024-04-25T17:08:11.149082+0000","last_unstale":"2024-04-25T17:08:13.352660+0000","last_undegraded":"2024-04-25T17:08:13.352660+0000","last_fullsized":"2024-04-25T17:08:13.352660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:14.565877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122122+0000","last_change":"2024-04-25T17:08:12.160331+0000","last_active":"2024-04-25T17:08:14.122122+0000","last_peered":"2024-04-25T17:08:14.122122+0000","last_clean":"2024-04-25T17:08:14.122122+0000","last_became_active":"2024-04-25T17:08:12.159931+0000","last_became_peered":"2024-04-25T17:08:12.159931+0000","last_unstale":"2024-04-25T17:08:14.122122+0000","last_undegraded":"2024-04-25T17:08:14.122122+0000","last_fullsized":"2024-04-25T17:08:14.122122+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:37.246974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.163558+0000","last_change":"2024-04-25T17:08:11.163118+0000","last_active":"2024-04-25T17:08:11.163558+0000","last_peered":"2024-04-25T17:08:11.163558+0000","last_clean":"2024-04-25T17:08:11.163558+0000","last_became_active":"2024-04-25T17:08:11.161691+0000","last_became_peered":"2024-04-25T17:08:11.161691+0000","last_unstale":"2024-04-25T17:08:11.163558+0000","last_undegraded":"2024-04-25T17:08:11.163558+0000","last_fullsized":"2024-04-25T17:08:11.163558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:23.135262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121927+0000","last_change":"2024-04-25T17:08:12.168387+0000","last_active":"2024-04-25T17:08:14.121927+0000","last_peered":"2024-04-25T17:08:14.121927+0000","last_clean":"2024-04-25T17:08:14.121927+0000","last_became_active":"2024-04-25T17:08:12.168008+0000","last_became_peered":"2024-04-25T17:08:12.168008+0000","last_unstale":"2024-04-25T17:08:14.121927+0000","last_undegraded":"2024-04-25T17:08:14.121927+0000","last_fullsized":"2024-04-25T17:08:14.121927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:08.438945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174728+0000","last_change":"2024-04-25T17:08:11.149181+0000","last_active":"2024-04-25T17:08:12.174728+0000","last_peered":"2024-04-25T17:08:12.174728+0000","last_clean":"2024-04-25T17:08:12.174728+0000","last_became_active":"2024-04-25T17:08:11.148847+0000","last_became_peered":"2024-04-25T17:08:11.148847+0000","last_unstale":"2024-04-25T17:08:12.174728+0000","last_undegraded":"2024-04-25T17:08:12.174728+0000","last_fullsized":"2024-04-25T17:08:12.174728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:40.720578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122115+0000","last_change":"2024-04-25T17:08:12.162173+0000","last_active":"2024-04-25T17:08:14.122115+0000","last_peered":"2024-04-25T17:08:14.122115+0000","last_clean":"2024-04-25T17:08:14.122115+0000","last_became_active":"2024-04-25T17:08:12.161818+0000","last_became_peered":"2024-04-25T17:08:12.161818+0000","last_unstale":"2024-04-25T17:08:14.122115+0000","last_undegraded":"2024-04-25T17:08:14.122115+0000","last_fullsized":"2024-04-25T17:08:14.122115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:28.872937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167954+0000","last_change":"2024-04-25T17:08:12.165542+0000","last_active":"2024-04-25T17:08:12.167954+0000","last_peered":"2024-04-25T17:08:12.167954+0000","last_clean":"2024-04-25T17:08:12.167954+0000","last_became_active":"2024-04-25T17:08:12.165190+0000","last_became_peered":"2024-04-25T17:08:12.165190+0000","last_unstale":"2024-04-25T17:08:12.167954+0000","last_undegraded":"2024-04-25T17:08:12.167954+0000","last_fullsized":"2024-04-25T17:08:12.167954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.414927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352355+0000","last_change":"2024-04-25T17:08:12.161429+0000","last_active":"2024-04-25T17:08:13.352355+0000","last_peered":"2024-04-25T17:08:13.352355+0000","last_clean":"2024-04-25T17:08:13.352355+0000","last_became_active":"2024-04-25T17:08:12.159184+0000","last_became_peered":"2024-04-25T17:08:12.159184+0000","last_unstale":"2024-04-25T17:08:13.352355+0000","last_undegraded":"2024-04-25T17:08:13.352355+0000","last_fullsized":"2024-04-25T17:08:13.352355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:56:21.366775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352298+0000","last_change":"2024-04-25T17:08:12.169948+0000","last_active":"2024-04-25T17:08:13.352298+0000","last_peered":"2024-04-25T17:08:13.352298+0000","last_clean":"2024-04-25T17:08:13.352298+0000","last_became_active":"2024-04-25T17:08:12.169592+0000","last_became_peered":"2024-04-25T17:08:12.169592+0000","last_unstale":"2024-04-25T17:08:13.352298+0000","last_undegraded":"2024-04-25T17:08:13.352298+0000","last_fullsized":"2024-04-25T17:08:13.352298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:22.749954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421852+0000","last_change":"2024-04-25T17:08:12.169785+0000","last_active":"2024-04-25T17:08:13.421852+0000","last_peered":"2024-04-25T17:08:13.421852+0000","last_clean":"2024-04-25T17:08:13.421852+0000","last_became_active":"2024-04-25T17:08:12.169226+0000","last_became_peered":"2024-04-25T17:08:12.169226+0000","last_unstale":"2024-04-25T17:08:13.421852+0000","last_undegraded":"2024-04-25T17:08:13.421852+0000","last_fullsized":"2024-04-25T17:08:13.421852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.232112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.162831+0000","last_change":"2024-04-25T17:08:12.157992+0000","last_active":"2024-04-25T17:08:12.162831+0000","last_peered":"2024-04-25T17:08:12.162831+0000","last_clean":"2024-04-25T17:08:12.162831+0000","last_became_active":"2024-04-25T17:08:12.157636+0000","last_became_peered":"2024-04-25T17:08:12.157636+0000","last_unstale":"2024-04-25T17:08:12.162831+0000","last_undegraded":"2024-04-25T17:08:12.162831+0000","last_fullsized":"2024-04-25T17:08:12.162831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:25.138871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.162331+0000","last_change":"2024-04-25T17:08:12.160977+0000","last_active":"2024-04-25T17:08:12.162331+0000","last_peered":"2024-04-25T17:08:12.162331+0000","last_clean":"2024-04-25T17:08:12.162331+0000","last_became_active":"2024-04-25T17:08:12.160571+0000","last_became_peered":"2024-04-25T17:08:12.160571+0000","last_unstale":"2024-04-25T17:08:12.162331+0000","last_undegraded":"2024-04-25T17:08:12.162331+0000","last_fullsized":"2024-04-25T17:08:12.162331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:13:47.932677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352838+0000","last_change":"2024-04-25T17:08:12.158566+0000","last_active":"2024-04-25T17:08:13.352838+0000","last_peered":"2024-04-25T17:08:13.352838+0000","last_clean":"2024-04-25T17:08:13.352838+0000","last_became_active":"2024-04-25T17:08:12.157911+0000","last_became_peered":"2024-04-25T17:08:12.157911+0000","last_unstale":"2024-04-25T17:08:13.352838+0000","last_undegraded":"2024-04-25T17:08:13.352838+0000","last_fullsized":"2024-04-25T17:08:13.352838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:49.665779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176417+0000","last_change":"2024-04-25T17:08:12.174931+0000","last_active":"2024-04-25T17:08:12.176417+0000","last_peered":"2024-04-25T17:08:12.176417+0000","last_clean":"2024-04-25T17:08:12.176417+0000","last_became_active":"2024-04-25T17:08:12.174571+0000","last_became_peered":"2024-04-25T17:08:12.174571+0000","last_unstale":"2024-04-25T17:08:12.176417+0000","last_undegraded":"2024-04-25T17:08:12.176417+0000","last_fullsized":"2024-04-25T17:08:12.176417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:40.372717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.147694+0000","last_change":"2024-04-25T17:08:11.147694+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.147694+0000","last_undegraded":"2024-04-25T17:08:11.147694+0000","last_fullsized":"2024-04-25T17:08:11.147694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176383+0000","last_change":"2024-04-25T17:08:12.174095+0000","last_active":"2024-04-25T17:08:12.176383+0000","last_peered":"2024-04-25T17:08:12.176383+0000","last_clean":"2024-04-25T17:08:12.176383+0000","last_became_active":"2024-04-25T17:08:12.172786+0000","last_became_peered":"2024-04-25T17:08:12.172786+0000","last_unstale":"2024-04-25T17:08:12.176383+0000","last_undegraded":"2024-04-25T17:08:12.176383+0000","last_fullsized":"2024-04-25T17:08:12.176383+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.264530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495785+0000","last_change":"2024-04-25T17:08:12.158772+0000","last_active":"2024-04-25T17:08:13.495785+0000","last_peered":"2024-04-25T17:08:13.495785+0000","last_clean":"2024-04-25T17:08:13.495785+0000","last_became_active":"2024-04-25T17:08:12.158417+0000","last_became_peered":"2024-04-25T17:08:12.158417+0000","last_unstale":"2024-04-25T17:08:13.495785+0000","last_undegraded":"2024-04-25T17:08:13.495785+0000","last_fullsized":"2024-04-25T17:08:13.495785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:21.837887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163235+0000","last_change":"2024-04-25T17:08:12.162384+0000","last_active":"2024-04-25T17:08:12.163235+0000","last_peered":"2024-04-25T17:08:12.163235+0000","last_clean":"2024-04-25T17:08:12.163235+0000","last_became_active":"2024-04-25T17:08:12.161997+0000","last_became_peered":"2024-04-25T17:08:12.161997+0000","last_unstale":"2024-04-25T17:08:12.163235+0000","last_undegraded":"2024-04-25T17:08:12.163235+0000","last_fullsized":"2024-04-25T17:08:12.163235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:19.759938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.160211+0000","last_change":"2024-04-25T17:08:12.159015+0000","last_active":"2024-04-25T17:08:12.160211+0000","last_peered":"2024-04-25T17:08:12.160211+0000","last_clean":"2024-04-25T17:08:12.160211+0000","last_became_active":"2024-04-25T17:08:12.157999+0000","last_became_peered":"2024-04-25T17:08:12.157999+0000","last_unstale":"2024-04-25T17:08:12.160211+0000","last_undegraded":"2024-04-25T17:08:12.160211+0000","last_fullsized":"2024-04-25T17:08:12.160211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:15.456131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280965,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":43629,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":371473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874258,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":981294,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":802816,"data_stored":381132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":819200,"data_stored":389724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":864256,"data_stored":980070,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":798720,"data_stored":380550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":811008,"data_stored":382162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":815104,"data_stored":389210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":410049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":371473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":802816,"data_stored":380572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":835584,"data_stored":961841,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38227,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T17:08:18.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T17:08:18.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 -- 172.21.15.96:0/417619614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c108900 msgr2=0x7f562c10ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 --2- 172.21.15.96:0/417619614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c108900 0x7f562c10ad20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5624009ca0 tx=0x7f562402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 -- 172.21.15.96:0/417619614 shutdown_connections 2024-04-25T17:08:18.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 --2- 172.21.15.96:0/417619614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c108900 0x7f562c10ad20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 --2- 172.21.15.96:0/417619614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c103c00 0x7f562c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 --2- 172.21.15.96:0/417619614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c1029c0 0x7f562c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 -- 172.21.15.96:0/417619614 >> 172.21.15.96:0/417619614 conn(0x7f562c0fdbd0 msgr2=0x7f562c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 -- 172.21.15.96:0/417619614 shutdown_connections 2024-04-25T17:08:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.361+0000 7f5633d6c640 1 -- 172.21.15.96:0/417619614 wait complete. 2024-04-25T17:08:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 Processor -- start 2024-04-25T17:08:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 -- start start 2024-04-25T17:08:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c1029c0 0x7f562c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c103c00 0x7f562c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c108900 0x7f562c1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f562c10e4f0 con 0x7f562c1029c0 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f562c10e370 con 0x7f562c103c00 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5633d6c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f562c10e670 con 0x7f562c108900 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c1029c0 0x7f562c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c1029c0 0x7f562c06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47636/0 (socket says 172.21.15.96:47636) 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 -- 172.21.15.96:0/886826778 learned_addr learned my addr 172.21.15.96:0/886826778 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5632309640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c108900 0x7f562c1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631307640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c103c00 0x7f562c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 -- 172.21.15.96:0/886826778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c108900 msgr2=0x7f562c1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c108900 0x7f562c1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 -- 172.21.15.96:0/886826778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c103c00 msgr2=0x7f562c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c103c00 0x7f562c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 -- 172.21.15.96:0/886826778 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c1a1c50 con 0x7f562c1029c0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631307640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c103c00 0x7f562c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5631b08640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c1029c0 0x7f562c06a920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c970 tx=0x7f561c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.362+0000 7f5622ffd640 1 -- 172.21.15.96:0/886826778 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c004220 con 0x7f562c1029c0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.363+0000 7f5622ffd640 1 -- 172.21.15.96:0/886826778 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f561c0043c0 con 0x7f562c1029c0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.363+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c1a1f40 con 0x7f562c1029c0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.363+0000 7f5622ffd640 1 -- 172.21.15.96:0/886826778 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c004a00 con 0x7f562c1029c0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.363+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c1a23a0 con 0x7f562c1029c0 2024-04-25T17:08:18.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.363+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562c107710 con 0x7f562c1029c0 2024-04-25T17:08:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.365+0000 7f5622ffd640 1 -- 172.21.15.96:0/886826778 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f561c0079b0 con 0x7f562c1029c0 2024-04-25T17:08:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.365+0000 7f5622ffd640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f56180afb80 0x7f56180b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.365+0000 7f5631307640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f56180afb80 0x7f56180b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.366+0000 7f5622ffd640 1 -- 172.21.15.96:0/886826778 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f561c0dee50 con 0x7f562c1029c0 2024-04-25T17:08:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.366+0000 7f5631307640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f56180afb80 0x7f56180b2090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f562c06a2c0 tx=0x7f5628005db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.369+0000 7f5622ffd640 1 -- 172.21.15.96:0/886826778 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f561c014030 con 0x7f562c1029c0 2024-04-25T17:08:18.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.544+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f562c062fe0 con 0x7f56180afb80 2024-04-25T17:08:18.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.551+0000 7f5622ffd640 1 -- 172.21.15.96:0/886826778 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258464 (secure 0 0 0) 0x7f562c062fe0 con 0x7f56180afb80 2024-04-25T17:08:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:18.555 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.554+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f56180afb80 msgr2=0x7f56180b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f56180afb80 0x7f56180b2090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f562c06a2c0 tx=0x7f5628005db0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c1029c0 msgr2=0x7f562c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c1029c0 0x7f562c06a920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c970 tx=0x7f561c00ce30 comp rx=0 tx=0).stop 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 shutdown_connections 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f56180afb80 0x7f56180b2090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c108900 0x7f562c1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f562c103c00 0x7f562c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 --2- 172.21.15.96:0/886826778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f562c1029c0 0x7f562c06a920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 >> 172.21.15.96:0/886826778 conn(0x7f562c0fdbd0 msgr2=0x7f562c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:18.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 shutdown_connections 2024-04-25T17:08:18.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.555+0000 7f5633d6c640 1 -- 172.21.15.96:0/886826778 wait complete. 2024-04-25T17:08:18.564 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:18 smithi096 ceph-mon[29131]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:18.564 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:18 smithi096 ceph-mon[29131]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T17:08:18.564 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:18 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2344384353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:18.565 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:18 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3819931771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:18.565 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-04-25T17:08:17.049972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":85,"num_read_kb":73,"num_write":162,"num_write_kb":2162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358832,"kb_used_data":9620,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582992,"statfs":{"total":1151940427776,"available":1151572983808,"internally_reserved":0,"allocated":9850880,"data_stored":6379550,"data_compressed":107190,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001410"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167139+0000","last_change":"2024-04-25T17:08:12.162430+0000","last_active":"2024-04-25T17:08:12.167139+0000","last_peered":"2024-04-25T17:08:12.167139+0000","last_clean":"2024-04-25T17:08:12.167139+0000","last_became_active":"2024-04-25T17:08:12.162052+0000","last_became_peered":"2024-04-25T17:08:12.162052+0000","last_unstale":"2024-04-25T17:08:12.167139+0000","last_undegraded":"2024-04-25T17:08:12.167139+0000","last_fullsized":"2024-04-25T17:08:12.167139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:14.457325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421957+0000","last_change":"2024-04-25T17:08:12.169139+0000","last_active":"2024-04-25T17:08:13.421957+0000","last_peered":"2024-04-25T17:08:13.421957+0000","last_clean":"2024-04-25T17:08:13.421957+0000","last_became_active":"2024-04-25T17:08:12.168715+0000","last_became_peered":"2024-04-25T17:08:12.168715+0000","last_unstale":"2024-04-25T17:08:13.421957+0000","last_undegraded":"2024-04-25T17:08:13.421957+0000","last_fullsized":"2024-04-25T17:08:13.421957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:57.357750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421733+0000","last_change":"2024-04-25T17:08:12.159141+0000","last_active":"2024-04-25T17:08:13.421733+0000","last_peered":"2024-04-25T17:08:13.421733+0000","last_clean":"2024-04-25T17:08:13.421733+0000","last_became_active":"2024-04-25T17:08:12.158796+0000","last_became_peered":"2024-04-25T17:08:12.158796+0000","last_unstale":"2024-04-25T17:08:13.421733+0000","last_undegraded":"2024-04-25T17:08:13.421733+0000","last_fullsized":"2024-04-25T17:08:13.421733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:55.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.161521+0000","last_change":"2024-04-25T17:08:12.160464+0000","last_active":"2024-04-25T17:08:12.161521+0000","last_peered":"2024-04-25T17:08:12.161521+0000","last_clean":"2024-04-25T17:08:12.161521+0000","last_became_active":"2024-04-25T17:08:12.160118+0000","last_became_peered":"2024-04-25T17:08:12.160118+0000","last_unstale":"2024-04-25T17:08:12.161521+0000","last_undegraded":"2024-04-25T17:08:12.161521+0000","last_fullsized":"2024-04-25T17:08:12.161521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:57.579423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495515+0000","last_change":"2024-04-25T17:08:12.174591+0000","last_active":"2024-04-25T17:08:13.495515+0000","last_peered":"2024-04-25T17:08:13.495515+0000","last_clean":"2024-04-25T17:08:13.495515+0000","last_became_active":"2024-04-25T17:08:12.174173+0000","last_became_peered":"2024-04-25T17:08:12.174173+0000","last_unstale":"2024-04-25T17:08:13.495515+0000","last_undegraded":"2024-04-25T17:08:13.495515+0000","last_fullsized":"2024-04-25T17:08:13.495515+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:12.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141095+0000","last_change":"2024-04-25T17:08:12.164230+0000","last_active":"2024-04-25T17:08:13.141095+0000","last_peered":"2024-04-25T17:08:13.141095+0000","last_clean":"2024-04-25T17:08:13.141095+0000","last_became_active":"2024-04-25T17:08:12.163868+0000","last_became_peered":"2024-04-25T17:08:12.163868+0000","last_unstale":"2024-04-25T17:08:13.141095+0000","last_undegraded":"2024-04-25T17:08:13.141095+0000","last_fullsized":"2024-04-25T17:08:13.141095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:12.604164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121611+0000","last_change":"2024-04-25T17:08:12.161627+0000","last_active":"2024-04-25T17:08:14.121611+0000","last_peered":"2024-04-25T17:08:14.121611+0000","last_clean":"2024-04-25T17:08:14.121611+0000","last_became_active":"2024-04-25T17:08:12.161146+0000","last_became_peered":"2024-04-25T17:08:12.161146+0000","last_unstale":"2024-04-25T17:08:14.121611+0000","last_undegraded":"2024-04-25T17:08:14.121611+0000","last_fullsized":"2024-04-25T17:08:14.121611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.972053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.166908+0000","last_change":"2024-04-25T17:08:12.165512+0000","last_active":"2024-04-25T17:08:12.166908+0000","last_peered":"2024-04-25T17:08:12.166908+0000","last_clean":"2024-04-25T17:08:12.166908+0000","last_became_active":"2024-04-25T17:08:12.164018+0000","last_became_peered":"2024-04-25T17:08:12.164018+0000","last_unstale":"2024-04-25T17:08:12.166908+0000","last_undegraded":"2024-04-25T17:08:12.166908+0000","last_fullsized":"2024-04-25T17:08:12.166908+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:45:29.650067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495771+0000","last_change":"2024-04-25T17:08:12.173313+0000","last_active":"2024-04-25T17:08:13.495771+0000","last_peered":"2024-04-25T17:08:13.495771+0000","last_clean":"2024-04-25T17:08:13.495771+0000","last_became_active":"2024-04-25T17:08:12.171269+0000","last_became_peered":"2024-04-25T17:08:12.171269+0000","last_unstale":"2024-04-25T17:08:13.495771+0000","last_undegraded":"2024-04-25T17:08:13.495771+0000","last_fullsized":"2024-04-25T17:08:13.495771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:39.565973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141559+0000","last_change":"2024-04-25T17:08:12.166114+0000","last_active":"2024-04-25T17:08:13.141559+0000","last_peered":"2024-04-25T17:08:13.141559+0000","last_clean":"2024-04-25T17:08:13.141559+0000","last_became_active":"2024-04-25T17:08:12.165737+0000","last_became_peered":"2024-04-25T17:08:12.165737+0000","last_unstale":"2024-04-25T17:08:13.141559+0000","last_undegraded":"2024-04-25T17:08:13.141559+0000","last_fullsized":"2024-04-25T17:08:13.141559+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:55.624239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122116+0000","last_change":"2024-04-25T17:08:12.160435+0000","last_active":"2024-04-25T17:08:14.122116+0000","last_peered":"2024-04-25T17:08:14.122116+0000","last_clean":"2024-04-25T17:08:14.122116+0000","last_became_active":"2024-04-25T17:08:12.160085+0000","last_became_peered":"2024-04-25T17:08:12.160085+0000","last_unstale":"2024-04-25T17:08:14.122116+0000","last_undegraded":"2024-04-25T17:08:14.122116+0000","last_fullsized":"2024-04-25T17:08:14.122116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.709567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174761+0000","last_change":"2024-04-25T17:08:12.173990+0000","last_active":"2024-04-25T17:08:12.174761+0000","last_peered":"2024-04-25T17:08:12.174761+0000","last_clean":"2024-04-25T17:08:12.174761+0000","last_became_active":"2024-04-25T17:08:12.172838+0000","last_became_peered":"2024-04-25T17:08:12.172838+0000","last_unstale":"2024-04-25T17:08:12.174761+0000","last_undegraded":"2024-04-25T17:08:12.174761+0000","last_fullsized":"2024-04-25T17:08:12.174761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:16.784335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.169084+0000","last_change":"2024-04-25T17:08:12.167579+0000","last_active":"2024-04-25T17:08:12.169084+0000","last_peered":"2024-04-25T17:08:12.169084+0000","last_clean":"2024-04-25T17:08:12.169084+0000","last_became_active":"2024-04-25T17:08:12.166583+0000","last_became_peered":"2024-04-25T17:08:12.166583+0000","last_unstale":"2024-04-25T17:08:12.169084+0000","last_undegraded":"2024-04-25T17:08:12.169084+0000","last_fullsized":"2024-04-25T17:08:12.169084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:55.855450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421592+0000","last_change":"2024-04-25T17:08:12.161263+0000","last_active":"2024-04-25T17:08:13.421592+0000","last_peered":"2024-04-25T17:08:13.421592+0000","last_clean":"2024-04-25T17:08:13.421592+0000","last_became_active":"2024-04-25T17:08:12.160078+0000","last_became_peered":"2024-04-25T17:08:12.160078+0000","last_unstale":"2024-04-25T17:08:13.421592+0000","last_undegraded":"2024-04-25T17:08:13.421592+0000","last_fullsized":"2024-04-25T17:08:13.421592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:56.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421594+0000","last_change":"2024-04-25T17:08:12.167318+0000","last_active":"2024-04-25T17:08:13.421594+0000","last_peered":"2024-04-25T17:08:13.421594+0000","last_clean":"2024-04-25T17:08:13.421594+0000","last_became_active":"2024-04-25T17:08:12.166964+0000","last_became_peered":"2024-04-25T17:08:12.166964+0000","last_unstale":"2024-04-25T17:08:13.421594+0000","last_undegraded":"2024-04-25T17:08:13.421594+0000","last_fullsized":"2024-04-25T17:08:13.421594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:25.967818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494787+0000","last_change":"2024-04-25T17:08:12.166417+0000","last_active":"2024-04-25T17:08:13.494787+0000","last_peered":"2024-04-25T17:08:13.494787+0000","last_clean":"2024-04-25T17:08:13.494787+0000","last_became_active":"2024-04-25T17:08:12.165928+0000","last_became_peered":"2024-04-25T17:08:12.165928+0000","last_unstale":"2024-04-25T17:08:13.494787+0000","last_undegraded":"2024-04-25T17:08:13.494787+0000","last_fullsized":"2024-04-25T17:08:13.494787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:24:22.359814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167674+0000","last_change":"2024-04-25T17:08:12.164952+0000","last_active":"2024-04-25T17:08:12.167674+0000","last_peered":"2024-04-25T17:08:12.167674+0000","last_clean":"2024-04-25T17:08:12.167674+0000","last_became_active":"2024-04-25T17:08:12.164595+0000","last_became_peered":"2024-04-25T17:08:12.164595+0000","last_unstale":"2024-04-25T17:08:12.167674+0000","last_undegraded":"2024-04-25T17:08:12.167674+0000","last_fullsized":"2024-04-25T17:08:12.167674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:48.456115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.154737+0000","last_change":"2024-04-25T17:08:12.153583+0000","last_active":"2024-04-25T17:08:12.154737+0000","last_peered":"2024-04-25T17:08:12.154737+0000","last_clean":"2024-04-25T17:08:12.154737+0000","last_became_active":"2024-04-25T17:08:12.153070+0000","last_became_peered":"2024-04-25T17:08:12.153070+0000","last_unstale":"2024-04-25T17:08:12.154737+0000","last_undegraded":"2024-04-25T17:08:12.154737+0000","last_fullsized":"2024-04-25T17:08:12.154737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352707+0000","last_change":"2024-04-25T17:08:12.166837+0000","last_active":"2024-04-25T17:08:13.352707+0000","last_peered":"2024-04-25T17:08:13.352707+0000","last_clean":"2024-04-25T17:08:13.352707+0000","last_became_active":"2024-04-25T17:08:12.166477+0000","last_became_peered":"2024-04-25T17:08:12.166477+0000","last_unstale":"2024-04-25T17:08:13.352707+0000","last_undegraded":"2024-04-25T17:08:13.352707+0000","last_fullsized":"2024-04-25T17:08:13.352707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:50.492333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.170659+0000","last_change":"2024-04-25T17:08:11.153164+0000","last_active":"2024-04-25T17:08:12.170659+0000","last_peered":"2024-04-25T17:08:12.170659+0000","last_clean":"2024-04-25T17:08:12.170659+0000","last_became_active":"2024-04-25T17:08:11.152189+0000","last_became_peered":"2024-04-25T17:08:11.152189+0000","last_unstale":"2024-04-25T17:08:12.170659+0000","last_undegraded":"2024-04-25T17:08:12.170659+0000","last_fullsized":"2024-04-25T17:08:12.170659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:30.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.178622+0000","last_change":"2024-04-25T17:08:12.178117+0000","last_active":"2024-04-25T17:08:12.178622+0000","last_peered":"2024-04-25T17:08:12.178622+0000","last_clean":"2024-04-25T17:08:12.178622+0000","last_became_active":"2024-04-25T17:08:12.177767+0000","last_became_peered":"2024-04-25T17:08:12.177767+0000","last_unstale":"2024-04-25T17:08:12.178622+0000","last_undegraded":"2024-04-25T17:08:12.178622+0000","last_fullsized":"2024-04-25T17:08:12.178622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:41.464565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.179579+0000","last_change":"2024-04-25T17:08:11.153996+0000","last_active":"2024-04-25T17:08:12.179579+0000","last_peered":"2024-04-25T17:08:12.179579+0000","last_clean":"2024-04-25T17:08:12.179579+0000","last_became_active":"2024-04-25T17:08:11.152993+0000","last_became_peered":"2024-04-25T17:08:11.152993+0000","last_unstale":"2024-04-25T17:08:12.179579+0000","last_undegraded":"2024-04-25T17:08:12.179579+0000","last_fullsized":"2024-04-25T17:08:12.179579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:33.116616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422141+0000","last_change":"2024-04-25T17:08:12.163707+0000","last_active":"2024-04-25T17:08:13.422141+0000","last_peered":"2024-04-25T17:08:13.422141+0000","last_clean":"2024-04-25T17:08:13.422141+0000","last_became_active":"2024-04-25T17:08:12.161868+0000","last_became_peered":"2024-04-25T17:08:12.161868+0000","last_unstale":"2024-04-25T17:08:13.422141+0000","last_undegraded":"2024-04-25T17:08:13.422141+0000","last_fullsized":"2024-04-25T17:08:13.422141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.333036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496262+0000","last_change":"2024-04-25T17:08:11.158101+0000","last_active":"2024-04-25T17:08:13.496262+0000","last_peered":"2024-04-25T17:08:13.496262+0000","last_clean":"2024-04-25T17:08:13.496262+0000","last_became_active":"2024-04-25T17:08:11.157747+0000","last_became_peered":"2024-04-25T17:08:11.157747+0000","last_unstale":"2024-04-25T17:08:13.496262+0000","last_undegraded":"2024-04-25T17:08:13.496262+0000","last_fullsized":"2024-04-25T17:08:13.496262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:03.840823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495644+0000","last_change":"2024-04-25T17:08:12.170278+0000","last_active":"2024-04-25T17:08:13.495644+0000","last_peered":"2024-04-25T17:08:13.495644+0000","last_clean":"2024-04-25T17:08:13.495644+0000","last_became_active":"2024-04-25T17:08:12.169219+0000","last_became_peered":"2024-04-25T17:08:12.169219+0000","last_unstale":"2024-04-25T17:08:13.495644+0000","last_undegraded":"2024-04-25T17:08:13.495644+0000","last_fullsized":"2024-04-25T17:08:13.495644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:35.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496183+0000","last_change":"2024-04-25T17:08:11.153167+0000","last_active":"2024-04-25T17:08:13.496183+0000","last_peered":"2024-04-25T17:08:13.496183+0000","last_clean":"2024-04-25T17:08:13.496183+0000","last_became_active":"2024-04-25T17:08:11.152772+0000","last_became_peered":"2024-04-25T17:08:11.152772+0000","last_unstale":"2024-04-25T17:08:13.496183+0000","last_undegraded":"2024-04-25T17:08:13.496183+0000","last_fullsized":"2024-04-25T17:08:13.496183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:57.198963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.146203+0000","last_change":"2024-04-25T17:08:11.146203+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.146203+0000","last_undegraded":"2024-04-25T17:08:11.146203+0000","last_fullsized":"2024-04-25T17:08:11.146203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.159382+0000","last_change":"2024-04-25T17:08:11.158564+0000","last_active":"2024-04-25T17:08:11.159382+0000","last_peered":"2024-04-25T17:08:11.159382+0000","last_clean":"2024-04-25T17:08:11.159382+0000","last_became_active":"2024-04-25T17:08:11.158143+0000","last_became_peered":"2024-04-25T17:08:11.158143+0000","last_unstale":"2024-04-25T17:08:11.159382+0000","last_undegraded":"2024-04-25T17:08:11.159382+0000","last_fullsized":"2024-04-25T17:08:11.159382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:44.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495878+0000","last_change":"2024-04-25T17:08:12.167584+0000","last_active":"2024-04-25T17:08:13.495878+0000","last_peered":"2024-04-25T17:08:13.495878+0000","last_clean":"2024-04-25T17:08:13.495878+0000","last_became_active":"2024-04-25T17:08:12.166717+0000","last_became_peered":"2024-04-25T17:08:12.166717+0000","last_unstale":"2024-04-25T17:08:13.495878+0000","last_undegraded":"2024-04-25T17:08:13.495878+0000","last_fullsized":"2024-04-25T17:08:13.495878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:38.047671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141009+0000","last_change":"2024-04-25T17:08:11.159254+0000","last_active":"2024-04-25T17:08:13.141009+0000","last_peered":"2024-04-25T17:08:13.141009+0000","last_clean":"2024-04-25T17:08:13.141009+0000","last_became_active":"2024-04-25T17:08:11.158864+0000","last_became_peered":"2024-04-25T17:08:11.158864+0000","last_unstale":"2024-04-25T17:08:13.141009+0000","last_undegraded":"2024-04-25T17:08:13.141009+0000","last_fullsized":"2024-04-25T17:08:13.141009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:17.795010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.144854+0000","last_change":"2024-04-25T17:08:11.144854+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.144854+0000","last_undegraded":"2024-04-25T17:08:11.144854+0000","last_fullsized":"2024-04-25T17:08:11.144854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.148562+0000","last_change":"2024-04-25T17:08:11.150980+0000","last_active":"2024-04-25T17:08:12.148562+0000","last_peered":"2024-04-25T17:08:12.148562+0000","last_clean":"2024-04-25T17:08:12.148562+0000","last_became_active":"2024-04-25T17:08:11.150580+0000","last_became_peered":"2024-04-25T17:08:11.150580+0000","last_unstale":"2024-04-25T17:08:12.148562+0000","last_undegraded":"2024-04-25T17:08:12.148562+0000","last_fullsized":"2024-04-25T17:08:12.148562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:59.539673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.175719+0000","last_change":"2024-04-25T17:08:12.171013+0000","last_active":"2024-04-25T17:08:12.175719+0000","last_peered":"2024-04-25T17:08:12.175719+0000","last_clean":"2024-04-25T17:08:12.175719+0000","last_became_active":"2024-04-25T17:08:12.169868+0000","last_became_peered":"2024-04-25T17:08:12.169868+0000","last_unstale":"2024-04-25T17:08:12.175719+0000","last_undegraded":"2024-04-25T17:08:12.175719+0000","last_fullsized":"2024-04-25T17:08:12.175719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:59.044063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.158357+0000","last_change":"2024-04-25T17:08:11.157618+0000","last_active":"2024-04-25T17:08:11.158357+0000","last_peered":"2024-04-25T17:08:11.158357+0000","last_clean":"2024-04-25T17:08:11.158357+0000","last_became_active":"2024-04-25T17:08:11.157052+0000","last_became_peered":"2024-04-25T17:08:11.157052+0000","last_unstale":"2024-04-25T17:08:11.158357+0000","last_undegraded":"2024-04-25T17:08:11.158357+0000","last_fullsized":"2024-04-25T17:08:11.158357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:09.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.151617+0000","last_change":"2024-04-25T17:08:12.151093+0000","last_active":"2024-04-25T17:08:12.151617+0000","last_peered":"2024-04-25T17:08:12.151617+0000","last_clean":"2024-04-25T17:08:12.151617+0000","last_became_active":"2024-04-25T17:08:12.150699+0000","last_became_peered":"2024-04-25T17:08:12.150699+0000","last_unstale":"2024-04-25T17:08:12.151617+0000","last_undegraded":"2024-04-25T17:08:12.151617+0000","last_fullsized":"2024-04-25T17:08:12.151617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:46.331493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141389+0000","last_change":"2024-04-25T17:08:11.157673+0000","last_active":"2024-04-25T17:08:13.141389+0000","last_peered":"2024-04-25T17:08:13.141389+0000","last_clean":"2024-04-25T17:08:13.141389+0000","last_became_active":"2024-04-25T17:08:11.157089+0000","last_became_peered":"2024-04-25T17:08:11.157089+0000","last_unstale":"2024-04-25T17:08:13.141389+0000","last_undegraded":"2024-04-25T17:08:13.141389+0000","last_fullsized":"2024-04-25T17:08:13.141389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:47.586003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495779+0000","last_change":"2024-04-25T17:08:12.170666+0000","last_active":"2024-04-25T17:08:13.495779+0000","last_peered":"2024-04-25T17:08:13.495779+0000","last_clean":"2024-04-25T17:08:13.495779+0000","last_became_active":"2024-04-25T17:08:12.169607+0000","last_became_peered":"2024-04-25T17:08:12.169607+0000","last_unstale":"2024-04-25T17:08:13.495779+0000","last_undegraded":"2024-04-25T17:08:13.495779+0000","last_fullsized":"2024-04-25T17:08:13.495779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121887+0000","last_change":"2024-04-25T17:08:11.153407+0000","last_active":"2024-04-25T17:08:14.121887+0000","last_peered":"2024-04-25T17:08:14.121887+0000","last_clean":"2024-04-25T17:08:14.121887+0000","last_became_active":"2024-04-25T17:08:11.152932+0000","last_became_peered":"2024-04-25T17:08:11.152932+0000","last_unstale":"2024-04-25T17:08:14.121887+0000","last_undegraded":"2024-04-25T17:08:14.121887+0000","last_fullsized":"2024-04-25T17:08:14.121887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:06.712620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422086+0000","last_change":"2024-04-25T17:08:12.173788+0000","last_active":"2024-04-25T17:08:13.422086+0000","last_peered":"2024-04-25T17:08:13.422086+0000","last_clean":"2024-04-25T17:08:13.422086+0000","last_became_active":"2024-04-25T17:08:12.172714+0000","last_became_peered":"2024-04-25T17:08:12.172714+0000","last_unstale":"2024-04-25T17:08:13.422086+0000","last_undegraded":"2024-04-25T17:08:13.422086+0000","last_fullsized":"2024-04-25T17:08:13.422086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:18.892187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.161140+0000","last_change":"2024-04-25T17:08:11.159802+0000","last_active":"2024-04-25T17:08:11.161140+0000","last_peered":"2024-04-25T17:08:11.161140+0000","last_clean":"2024-04-25T17:08:11.161140+0000","last_became_active":"2024-04-25T17:08:11.159431+0000","last_became_peered":"2024-04-25T17:08:11.159431+0000","last_unstale":"2024-04-25T17:08:11.161140+0000","last_undegraded":"2024-04-25T17:08:11.161140+0000","last_fullsized":"2024-04-25T17:08:11.161140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:45.961985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495822+0000","last_change":"2024-04-25T17:08:12.162703+0000","last_active":"2024-04-25T17:08:13.495822+0000","last_peered":"2024-04-25T17:08:13.495822+0000","last_clean":"2024-04-25T17:08:13.495822+0000","last_became_active":"2024-04-25T17:08:12.162361+0000","last_became_peered":"2024-04-25T17:08:12.162361+0000","last_unstale":"2024-04-25T17:08:13.495822+0000","last_undegraded":"2024-04-25T17:08:13.495822+0000","last_fullsized":"2024-04-25T17:08:13.495822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:51.262319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'6","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141260+0000","last_change":"2024-04-25T17:08:11.153938+0000","last_active":"2024-04-25T17:08:13.141260+0000","last_peered":"2024-04-25T17:08:13.141260+0000","last_clean":"2024-04-25T17:08:13.141260+0000","last_became_active":"2024-04-25T17:08:11.153591+0000","last_became_peered":"2024-04-25T17:08:11.153591+0000","last_unstale":"2024-04-25T17:08:13.141260+0000","last_undegraded":"2024-04-25T17:08:13.141260+0000","last_fullsized":"2024-04-25T17:08:13.141260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496204+0000","last_change":"2024-04-25T17:08:12.156045+0000","last_active":"2024-04-25T17:08:13.496204+0000","last_peered":"2024-04-25T17:08:13.496204+0000","last_clean":"2024-04-25T17:08:13.496204+0000","last_became_active":"2024-04-25T17:08:12.155657+0000","last_became_peered":"2024-04-25T17:08:12.155657+0000","last_unstale":"2024-04-25T17:08:13.496204+0000","last_undegraded":"2024-04-25T17:08:13.496204+0000","last_fullsized":"2024-04-25T17:08:13.496204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352496+0000","last_change":"2024-04-25T17:08:11.162009+0000","last_active":"2024-04-25T17:08:13.352496+0000","last_peered":"2024-04-25T17:08:13.352496+0000","last_clean":"2024-04-25T17:08:13.352496+0000","last_became_active":"2024-04-25T17:08:11.161574+0000","last_became_peered":"2024-04-25T17:08:11.161574+0000","last_unstale":"2024-04-25T17:08:13.352496+0000","last_undegraded":"2024-04-25T17:08:13.352496+0000","last_fullsized":"2024-04-25T17:08:13.352496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:16.760355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163159+0000","last_change":"2024-04-25T17:08:12.159522+0000","last_active":"2024-04-25T17:08:12.163159+0000","last_peered":"2024-04-25T17:08:12.163159+0000","last_clean":"2024-04-25T17:08:12.163159+0000","last_became_active":"2024-04-25T17:08:12.159036+0000","last_became_peered":"2024-04-25T17:08:12.159036+0000","last_unstale":"2024-04-25T17:08:12.163159+0000","last_undegraded":"2024-04-25T17:08:12.163159+0000","last_fullsized":"2024-04-25T17:08:12.163159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:18.534810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.162996+0000","last_change":"2024-04-25T17:08:11.162487+0000","last_active":"2024-04-25T17:08:11.162996+0000","last_peered":"2024-04-25T17:08:11.162996+0000","last_clean":"2024-04-25T17:08:11.162996+0000","last_became_active":"2024-04-25T17:08:11.161961+0000","last_became_peered":"2024-04-25T17:08:11.161961+0000","last_unstale":"2024-04-25T17:08:11.162996+0000","last_undegraded":"2024-04-25T17:08:11.162996+0000","last_fullsized":"2024-04-25T17:08:11.162996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:09.095387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121943+0000","last_change":"2024-04-25T17:08:12.164107+0000","last_active":"2024-04-25T17:08:14.121943+0000","last_peered":"2024-04-25T17:08:14.121943+0000","last_clean":"2024-04-25T17:08:14.121943+0000","last_became_active":"2024-04-25T17:08:12.163741+0000","last_became_peered":"2024-04-25T17:08:12.163741+0000","last_unstale":"2024-04-25T17:08:14.121943+0000","last_undegraded":"2024-04-25T17:08:14.121943+0000","last_fullsized":"2024-04-25T17:08:14.121943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.552833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.146579+0000","last_change":"2024-04-25T17:08:11.153383+0000","last_active":"2024-04-25T17:08:12.146579+0000","last_peered":"2024-04-25T17:08:12.146579+0000","last_clean":"2024-04-25T17:08:12.146579+0000","last_became_active":"2024-04-25T17:08:11.152573+0000","last_became_peered":"2024-04-25T17:08:11.152573+0000","last_unstale":"2024-04-25T17:08:12.146579+0000","last_undegraded":"2024-04-25T17:08:12.146579+0000","last_fullsized":"2024-04-25T17:08:12.146579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:13.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121535+0000","last_change":"2024-04-25T17:08:12.165939+0000","last_active":"2024-04-25T17:08:14.121535+0000","last_peered":"2024-04-25T17:08:14.121535+0000","last_clean":"2024-04-25T17:08:14.121535+0000","last_became_active":"2024-04-25T17:08:12.165569+0000","last_became_peered":"2024-04-25T17:08:12.165569+0000","last_unstale":"2024-04-25T17:08:14.121535+0000","last_undegraded":"2024-04-25T17:08:14.121535+0000","last_fullsized":"2024-04-25T17:08:14.121535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:15:16.230457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495142+0000","last_change":"2024-04-25T17:08:11.158927+0000","last_active":"2024-04-25T17:08:13.495142+0000","last_peered":"2024-04-25T17:08:13.495142+0000","last_clean":"2024-04-25T17:08:13.495142+0000","last_became_active":"2024-04-25T17:08:11.158528+0000","last_became_peered":"2024-04-25T17:08:11.158528+0000","last_unstale":"2024-04-25T17:08:13.495142+0000","last_undegraded":"2024-04-25T17:08:13.495142+0000","last_fullsized":"2024-04-25T17:08:13.495142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.308627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.143338+0000","last_change":"2024-04-25T17:08:11.143338+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.143338+0000","last_undegraded":"2024-04-25T17:08:11.143338+0000","last_fullsized":"2024-04-25T17:08:11.143338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495648+0000","last_change":"2024-04-25T17:08:11.153214+0000","last_active":"2024-04-25T17:08:13.495648+0000","last_peered":"2024-04-25T17:08:13.495648+0000","last_clean":"2024-04-25T17:08:13.495648+0000","last_became_active":"2024-04-25T17:08:11.152728+0000","last_became_peered":"2024-04-25T17:08:11.152728+0000","last_unstale":"2024-04-25T17:08:13.495648+0000","last_undegraded":"2024-04-25T17:08:13.495648+0000","last_fullsized":"2024-04-25T17:08:13.495648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.338549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167104+0000","last_change":"2024-04-25T17:08:12.162167+0000","last_active":"2024-04-25T17:08:12.167104+0000","last_peered":"2024-04-25T17:08:12.167104+0000","last_clean":"2024-04-25T17:08:12.167104+0000","last_became_active":"2024-04-25T17:08:12.160896+0000","last_became_peered":"2024-04-25T17:08:12.160896+0000","last_unstale":"2024-04-25T17:08:12.167104+0000","last_undegraded":"2024-04-25T17:08:12.167104+0000","last_fullsized":"2024-04-25T17:08:12.167104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:20.437234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.141261+0000","last_change":"2024-04-25T17:08:11.158995+0000","last_active":"2024-04-25T17:08:12.141261+0000","last_peered":"2024-04-25T17:08:12.141261+0000","last_clean":"2024-04-25T17:08:12.141261+0000","last_became_active":"2024-04-25T17:08:11.158644+0000","last_became_peered":"2024-04-25T17:08:11.158644+0000","last_unstale":"2024-04-25T17:08:12.141261+0000","last_undegraded":"2024-04-25T17:08:12.141261+0000","last_fullsized":"2024-04-25T17:08:12.141261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:22.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141392+0000","last_change":"2024-04-25T17:08:12.161635+0000","last_active":"2024-04-25T17:08:13.141392+0000","last_peered":"2024-04-25T17:08:13.141392+0000","last_clean":"2024-04-25T17:08:13.141392+0000","last_became_active":"2024-04-25T17:08:12.153728+0000","last_became_peered":"2024-04-25T17:08:12.153728+0000","last_unstale":"2024-04-25T17:08:13.141392+0000","last_undegraded":"2024-04-25T17:08:13.141392+0000","last_fullsized":"2024-04-25T17:08:13.141392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:02.003550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121630+0000","last_change":"2024-04-25T17:08:11.150490+0000","last_active":"2024-04-25T17:08:14.121630+0000","last_peered":"2024-04-25T17:08:14.121630+0000","last_clean":"2024-04-25T17:08:14.121630+0000","last_became_active":"2024-04-25T17:08:11.148391+0000","last_became_peered":"2024-04-25T17:08:11.148391+0000","last_unstale":"2024-04-25T17:08:14.121630+0000","last_undegraded":"2024-04-25T17:08:14.121630+0000","last_fullsized":"2024-04-25T17:08:14.121630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:40.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141295+0000","last_change":"2024-04-25T17:08:12.168136+0000","last_active":"2024-04-25T17:08:13.141295+0000","last_peered":"2024-04-25T17:08:13.141295+0000","last_clean":"2024-04-25T17:08:13.141295+0000","last_became_active":"2024-04-25T17:08:12.167786+0000","last_became_peered":"2024-04-25T17:08:12.167786+0000","last_unstale":"2024-04-25T17:08:13.141295+0000","last_undegraded":"2024-04-25T17:08:13.141295+0000","last_fullsized":"2024-04-25T17:08:13.141295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:22.869503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.149194+0000","last_change":"2024-04-25T17:08:11.149194+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.149194+0000","last_undegraded":"2024-04-25T17:08:11.149194+0000","last_fullsized":"2024-04-25T17:08:11.149194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.149557+0000","last_change":"2024-04-25T17:08:11.149085+0000","last_active":"2024-04-25T17:08:12.149557+0000","last_peered":"2024-04-25T17:08:12.149557+0000","last_clean":"2024-04-25T17:08:12.149557+0000","last_became_active":"2024-04-25T17:08:11.148712+0000","last_became_peered":"2024-04-25T17:08:11.148712+0000","last_unstale":"2024-04-25T17:08:12.149557+0000","last_undegraded":"2024-04-25T17:08:12.149557+0000","last_fullsized":"2024-04-25T17:08:12.149557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:12.764034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.178626+0000","last_change":"2024-04-25T17:08:12.178154+0000","last_active":"2024-04-25T17:08:12.178626+0000","last_peered":"2024-04-25T17:08:12.178626+0000","last_clean":"2024-04-25T17:08:12.178626+0000","last_became_active":"2024-04-25T17:08:12.177773+0000","last_became_peered":"2024-04-25T17:08:12.177773+0000","last_unstale":"2024-04-25T17:08:12.178626+0000","last_undegraded":"2024-04-25T17:08:12.178626+0000","last_fullsized":"2024-04-25T17:08:12.178626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:30.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.148011+0000","last_change":"2024-04-25T17:08:11.152932+0000","last_active":"2024-04-25T17:08:12.148011+0000","last_peered":"2024-04-25T17:08:12.148011+0000","last_clean":"2024-04-25T17:08:12.148011+0000","last_became_active":"2024-04-25T17:08:11.152530+0000","last_became_peered":"2024-04-25T17:08:11.152530+0000","last_unstale":"2024-04-25T17:08:12.148011+0000","last_undegraded":"2024-04-25T17:08:12.148011+0000","last_fullsized":"2024-04-25T17:08:12.148011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121425+0000","last_change":"2024-04-25T17:06:55.951326+0000","last_active":"2024-04-25T17:08:14.121425+0000","last_peered":"2024-04-25T17:08:14.121425+0000","last_clean":"2024-04-25T17:08:14.121425+0000","last_became_active":"2024-04-25T17:06:55.946641+0000","last_became_peered":"2024-04-25T17:06:55.946641+0000","last_unstale":"2024-04-25T17:08:14.121425+0000","last_undegraded":"2024-04-25T17:08:14.121425+0000","last_fullsized":"2024-04-25T17:08:14.121425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_clean_scrub_stamp":"2024-04-25T17:03:27.394276+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:58.308843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163189+0000","last_change":"2024-04-25T17:08:12.162065+0000","last_active":"2024-04-25T17:08:12.163189+0000","last_peered":"2024-04-25T17:08:12.163189+0000","last_clean":"2024-04-25T17:08:12.163189+0000","last_became_active":"2024-04-25T17:08:12.161733+0000","last_became_peered":"2024-04-25T17:08:12.161733+0000","last_unstale":"2024-04-25T17:08:12.163189+0000","last_undegraded":"2024-04-25T17:08:12.163189+0000","last_fullsized":"2024-04-25T17:08:12.163189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:05.132252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121612+0000","last_change":"2024-04-25T17:08:11.155941+0000","last_active":"2024-04-25T17:08:14.121612+0000","last_peered":"2024-04-25T17:08:14.121612+0000","last_clean":"2024-04-25T17:08:14.121612+0000","last_became_active":"2024-04-25T17:08:11.155381+0000","last_became_peered":"2024-04-25T17:08:11.155381+0000","last_unstale":"2024-04-25T17:08:14.121612+0000","last_undegraded":"2024-04-25T17:08:14.121612+0000","last_fullsized":"2024-04-25T17:08:14.121612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:28.423462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496103+0000","last_change":"2024-04-25T17:08:11.158960+0000","last_active":"2024-04-25T17:08:13.496103+0000","last_peered":"2024-04-25T17:08:13.496103+0000","last_clean":"2024-04-25T17:08:13.496103+0000","last_became_active":"2024-04-25T17:08:11.158609+0000","last_became_peered":"2024-04-25T17:08:11.158609+0000","last_unstale":"2024-04-25T17:08:13.496103+0000","last_undegraded":"2024-04-25T17:08:13.496103+0000","last_fullsized":"2024-04-25T17:08:13.496103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:19.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163020+0000","last_change":"2024-04-25T17:08:12.161483+0000","last_active":"2024-04-25T17:08:12.163020+0000","last_peered":"2024-04-25T17:08:12.163020+0000","last_clean":"2024-04-25T17:08:12.163020+0000","last_became_active":"2024-04-25T17:08:12.158586+0000","last_became_peered":"2024-04-25T17:08:12.158586+0000","last_unstale":"2024-04-25T17:08:12.163020+0000","last_undegraded":"2024-04-25T17:08:12.163020+0000","last_fullsized":"2024-04-25T17:08:12.163020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.141542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.141400+0000","last_change":"2024-04-25T17:08:11.161497+0000","last_active":"2024-04-25T17:08:12.141400+0000","last_peered":"2024-04-25T17:08:12.141400+0000","last_clean":"2024-04-25T17:08:12.141400+0000","last_became_active":"2024-04-25T17:08:11.160990+0000","last_became_peered":"2024-04-25T17:08:11.160990+0000","last_unstale":"2024-04-25T17:08:12.141400+0000","last_undegraded":"2024-04-25T17:08:12.141400+0000","last_fullsized":"2024-04-25T17:08:12.141400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:59.682133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176351+0000","last_change":"2024-04-25T17:08:12.171214+0000","last_active":"2024-04-25T17:08:12.176351+0000","last_peered":"2024-04-25T17:08:12.176351+0000","last_clean":"2024-04-25T17:08:12.176351+0000","last_became_active":"2024-04-25T17:08:12.170877+0000","last_became_peered":"2024-04-25T17:08:12.170877+0000","last_unstale":"2024-04-25T17:08:12.176351+0000","last_undegraded":"2024-04-25T17:08:12.176351+0000","last_fullsized":"2024-04-25T17:08:12.176351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:18.267340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176348+0000","last_change":"2024-04-25T17:08:11.154819+0000","last_active":"2024-04-25T17:08:12.176348+0000","last_peered":"2024-04-25T17:08:12.176348+0000","last_clean":"2024-04-25T17:08:12.176348+0000","last_became_active":"2024-04-25T17:08:11.154006+0000","last_became_peered":"2024-04-25T17:08:11.154006+0000","last_unstale":"2024-04-25T17:08:12.176348+0000","last_undegraded":"2024-04-25T17:08:12.176348+0000","last_fullsized":"2024-04-25T17:08:12.176348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.063058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422056+0000","last_change":"2024-04-25T17:08:12.167301+0000","last_active":"2024-04-25T17:08:13.422056+0000","last_peered":"2024-04-25T17:08:13.422056+0000","last_clean":"2024-04-25T17:08:13.422056+0000","last_became_active":"2024-04-25T17:08:12.166944+0000","last_became_peered":"2024-04-25T17:08:12.166944+0000","last_unstale":"2024-04-25T17:08:13.422056+0000","last_undegraded":"2024-04-25T17:08:13.422056+0000","last_fullsized":"2024-04-25T17:08:13.422056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:46.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495108+0000","last_change":"2024-04-25T17:08:11.156616+0000","last_active":"2024-04-25T17:08:13.495108+0000","last_peered":"2024-04-25T17:08:13.495108+0000","last_clean":"2024-04-25T17:08:13.495108+0000","last_became_active":"2024-04-25T17:08:11.156256+0000","last_became_peered":"2024-04-25T17:08:11.156256+0000","last_unstale":"2024-04-25T17:08:13.495108+0000","last_undegraded":"2024-04-25T17:08:13.495108+0000","last_fullsized":"2024-04-25T17:08:13.495108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:37.539773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495319+0000","last_change":"2024-04-25T17:08:12.175216+0000","last_active":"2024-04-25T17:08:13.495319+0000","last_peered":"2024-04-25T17:08:13.495319+0000","last_clean":"2024-04-25T17:08:13.495319+0000","last_became_active":"2024-04-25T17:08:12.174521+0000","last_became_peered":"2024-04-25T17:08:12.174521+0000","last_unstale":"2024-04-25T17:08:13.495319+0000","last_undegraded":"2024-04-25T17:08:13.495319+0000","last_fullsized":"2024-04-25T17:08:13.495319+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.322733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.152221+0000","last_change":"2024-04-25T17:08:11.162287+0000","last_active":"2024-04-25T17:08:12.152221+0000","last_peered":"2024-04-25T17:08:12.152221+0000","last_clean":"2024-04-25T17:08:12.152221+0000","last_became_active":"2024-04-25T17:08:11.158297+0000","last_became_peered":"2024-04-25T17:08:11.158297+0000","last_unstale":"2024-04-25T17:08:12.152221+0000","last_undegraded":"2024-04-25T17:08:12.152221+0000","last_fullsized":"2024-04-25T17:08:12.152221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:15.758880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167929+0000","last_change":"2024-04-25T17:08:12.163535+0000","last_active":"2024-04-25T17:08:12.167929+0000","last_peered":"2024-04-25T17:08:12.167929+0000","last_clean":"2024-04-25T17:08:12.167929+0000","last_became_active":"2024-04-25T17:08:12.162848+0000","last_became_peered":"2024-04-25T17:08:12.162848+0000","last_unstale":"2024-04-25T17:08:12.167929+0000","last_undegraded":"2024-04-25T17:08:12.167929+0000","last_fullsized":"2024-04-25T17:08:12.167929+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:36.839617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352187+0000","last_change":"2024-04-25T17:08:11.162780+0000","last_active":"2024-04-25T17:08:13.352187+0000","last_peered":"2024-04-25T17:08:13.352187+0000","last_clean":"2024-04-25T17:08:13.352187+0000","last_became_active":"2024-04-25T17:08:11.162385+0000","last_became_peered":"2024-04-25T17:08:11.162385+0000","last_unstale":"2024-04-25T17:08:13.352187+0000","last_undegraded":"2024-04-25T17:08:13.352187+0000","last_fullsized":"2024-04-25T17:08:13.352187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:25.420740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174051+0000","last_change":"2024-04-25T17:08:12.170828+0000","last_active":"2024-04-25T17:08:12.174051+0000","last_peered":"2024-04-25T17:08:12.174051+0000","last_clean":"2024-04-25T17:08:12.174051+0000","last_became_active":"2024-04-25T17:08:12.170479+0000","last_became_peered":"2024-04-25T17:08:12.170479+0000","last_unstale":"2024-04-25T17:08:12.174051+0000","last_undegraded":"2024-04-25T17:08:12.174051+0000","last_fullsized":"2024-04-25T17:08:12.174051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:30.418656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495662+0000","last_change":"2024-04-25T17:08:11.156754+0000","last_active":"2024-04-25T17:08:13.495662+0000","last_peered":"2024-04-25T17:08:13.495662+0000","last_clean":"2024-04-25T17:08:13.495662+0000","last_became_active":"2024-04-25T17:08:11.156414+0000","last_became_peered":"2024-04-25T17:08:11.156414+0000","last_unstale":"2024-04-25T17:08:13.495662+0000","last_undegraded":"2024-04-25T17:08:13.495662+0000","last_fullsized":"2024-04-25T17:08:13.495662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:10.364234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121978+0000","last_change":"2024-04-25T17:08:12.162032+0000","last_active":"2024-04-25T17:08:14.121978+0000","last_peered":"2024-04-25T17:08:14.121978+0000","last_clean":"2024-04-25T17:08:14.121978+0000","last_became_active":"2024-04-25T17:08:12.161017+0000","last_became_peered":"2024-04-25T17:08:12.161017+0000","last_unstale":"2024-04-25T17:08:14.121978+0000","last_undegraded":"2024-04-25T17:08:14.121978+0000","last_fullsized":"2024-04-25T17:08:14.121978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.472984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352660+0000","last_change":"2024-04-25T17:08:11.149550+0000","last_active":"2024-04-25T17:08:13.352660+0000","last_peered":"2024-04-25T17:08:13.352660+0000","last_clean":"2024-04-25T17:08:13.352660+0000","last_became_active":"2024-04-25T17:08:11.149082+0000","last_became_peered":"2024-04-25T17:08:11.149082+0000","last_unstale":"2024-04-25T17:08:13.352660+0000","last_undegraded":"2024-04-25T17:08:13.352660+0000","last_fullsized":"2024-04-25T17:08:13.352660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:14.565877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122122+0000","last_change":"2024-04-25T17:08:12.160331+0000","last_active":"2024-04-25T17:08:14.122122+0000","last_peered":"2024-04-25T17:08:14.122122+0000","last_clean":"2024-04-25T17:08:14.122122+0000","last_became_active":"2024-04-25T17:08:12.159931+0000","last_became_peered":"2024-04-25T17:08:12.159931+0000","last_unstale":"2024-04-25T17:08:14.122122+0000","last_undegraded":"2024-04-25T17:08:14.122122+0000","last_fullsized":"2024-04-25T17:08:14.122122+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:37.246974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.163558+0000","last_change":"2024-04-25T17:08:11.163118+0000","last_active":"2024-04-25T17:08:11.163558+0000","last_peered":"2024-04-25T17:08:11.163558+0000","last_clean":"2024-04-25T17:08:11.163558+0000","last_became_active":"2024-04-25T17:08:11.161691+0000","last_became_peered":"2024-04-25T17:08:11.161691+0000","last_unstale":"2024-04-25T17:08:11.163558+0000","last_undegraded":"2024-04-25T17:08:11.163558+0000","last_fullsized":"2024-04-25T17:08:11.163558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:23.135262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121927+0000","last_change":"2024-04-25T17:08:12.168387+0000","last_active":"2024-04-25T17:08:14.121927+0000","last_peered":"2024-04-25T17:08:14.121927+0000","last_clean":"2024-04-25T17:08:14.121927+0000","last_became_active":"2024-04-25T17:08:12.168008+0000","last_became_peered":"2024-04-25T17:08:12.168008+0000","last_unstale":"2024-04-25T17:08:14.121927+0000","last_undegraded":"2024-04-25T17:08:14.121927+0000","last_fullsized":"2024-04-25T17:08:14.121927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:08.438945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174728+0000","last_change":"2024-04-25T17:08:11.149181+0000","last_active":"2024-04-25T17:08:12.174728+0000","last_peered":"2024-04-25T17:08:12.174728+0000","last_clean":"2024-04-25T17:08:12.174728+0000","last_became_active":"2024-04-25T17:08:11.148847+0000","last_became_peered":"2024-04-25T17:08:11.148847+0000","last_unstale":"2024-04-25T17:08:12.174728+0000","last_undegraded":"2024-04-25T17:08:12.174728+0000","last_fullsized":"2024-04-25T17:08:12.174728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:40.720578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122115+0000","last_change":"2024-04-25T17:08:12.162173+0000","last_active":"2024-04-25T17:08:14.122115+0000","last_peered":"2024-04-25T17:08:14.122115+0000","last_clean":"2024-04-25T17:08:14.122115+0000","last_became_active":"2024-04-25T17:08:12.161818+0000","last_became_peered":"2024-04-25T17:08:12.161818+0000","last_unstale":"2024-04-25T17:08:14.122115+0000","last_undegraded":"2024-04-25T17:08:14.122115+0000","last_fullsized":"2024-04-25T17:08:14.122115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:28.872937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167954+0000","last_change":"2024-04-25T17:08:12.165542+0000","last_active":"2024-04-25T17:08:12.167954+0000","last_peered":"2024-04-25T17:08:12.167954+0000","last_clean":"2024-04-25T17:08:12.167954+0000","last_became_active":"2024-04-25T17:08:12.165190+0000","last_became_peered":"2024-04-25T17:08:12.165190+0000","last_unstale":"2024-04-25T17:08:12.167954+0000","last_undegraded":"2024-04-25T17:08:12.167954+0000","last_fullsized":"2024-04-25T17:08:12.167954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.414927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352355+0000","last_change":"2024-04-25T17:08:12.161429+0000","last_active":"2024-04-25T17:08:13.352355+0000","last_peered":"2024-04-25T17:08:13.352355+0000","last_clean":"2024-04-25T17:08:13.352355+0000","last_became_active":"2024-04-25T17:08:12.159184+0000","last_became_peered":"2024-04-25T17:08:12.159184+0000","last_unstale":"2024-04-25T17:08:13.352355+0000","last_undegraded":"2024-04-25T17:08:13.352355+0000","last_fullsized":"2024-04-25T17:08:13.352355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:56:21.366775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352298+0000","last_change":"2024-04-25T17:08:12.169948+0000","last_active":"2024-04-25T17:08:13.352298+0000","last_peered":"2024-04-25T17:08:13.352298+0000","last_clean":"2024-04-25T17:08:13.352298+0000","last_became_active":"2024-04-25T17:08:12.169592+0000","last_became_peered":"2024-04-25T17:08:12.169592+0000","last_unstale":"2024-04-25T17:08:13.352298+0000","last_undegraded":"2024-04-25T17:08:13.352298+0000","last_fullsized":"2024-04-25T17:08:13.352298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:22.749954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421852+0000","last_change":"2024-04-25T17:08:12.169785+0000","last_active":"2024-04-25T17:08:13.421852+0000","last_peered":"2024-04-25T17:08:13.421852+0000","last_clean":"2024-04-25T17:08:13.421852+0000","last_became_active":"2024-04-25T17:08:12.169226+0000","last_became_peered":"2024-04-25T17:08:12.169226+0000","last_unstale":"2024-04-25T17:08:13.421852+0000","last_undegraded":"2024-04-25T17:08:13.421852+0000","last_fullsized":"2024-04-25T17:08:13.421852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.232112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.162831+0000","last_change":"2024-04-25T17:08:12.157992+0000","last_active":"2024-04-25T17:08:12.162831+0000","last_peered":"2024-04-25T17:08:12.162831+0000","last_clean":"2024-04-25T17:08:12.162831+0000","last_became_active":"2024-04-25T17:08:12.157636+0000","last_became_peered":"2024-04-25T17:08:12.157636+0000","last_unstale":"2024-04-25T17:08:12.162831+0000","last_undegraded":"2024-04-25T17:08:12.162831+0000","last_fullsized":"2024-04-25T17:08:12.162831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:25.138871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.162331+0000","last_change":"2024-04-25T17:08:12.160977+0000","last_active":"2024-04-25T17:08:12.162331+0000","last_peered":"2024-04-25T17:08:12.162331+0000","last_clean":"2024-04-25T17:08:12.162331+0000","last_became_active":"2024-04-25T17:08:12.160571+0000","last_became_peered":"2024-04-25T17:08:12.160571+0000","last_unstale":"2024-04-25T17:08:12.162331+0000","last_undegraded":"2024-04-25T17:08:12.162331+0000","last_fullsized":"2024-04-25T17:08:12.162331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:13:47.932677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352838+0000","last_change":"2024-04-25T17:08:12.158566+0000","last_active":"2024-04-25T17:08:13.352838+0000","last_peered":"2024-04-25T17:08:13.352838+0000","last_clean":"2024-04-25T17:08:13.352838+0000","last_became_active":"2024-04-25T17:08:12.157911+0000","last_became_peered":"2024-04-25T17:08:12.157911+0000","last_unstale":"2024-04-25T17:08:13.352838+0000","last_undegraded":"2024-04-25T17:08:13.352838+0000","last_fullsized":"2024-04-25T17:08:13.352838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:49.665779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176417+0000","last_change":"2024-04-25T17:08:12.174931+0000","last_active":"2024-04-25T17:08:12.176417+0000","last_peered":"2024-04-25T17:08:12.176417+0000","last_clean":"2024-04-25T17:08:12.176417+0000","last_became_active":"2024-04-25T17:08:12.174571+0000","last_became_peered":"2024-04-25T17:08:12.174571+0000","last_unstale":"2024-04-25T17:08:12.176417+0000","last_undegraded":"2024-04-25T17:08:12.176417+0000","last_fullsized":"2024-04-25T17:08:12.176417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:40.372717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.147694+0000","last_change":"2024-04-25T17:08:11.147694+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.147694+0000","last_undegraded":"2024-04-25T17:08:11.147694+0000","last_fullsized":"2024-04-25T17:08:11.147694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176383+0000","last_change":"2024-04-25T17:08:12.174095+0000","last_active":"2024-04-25T17:08:12.176383+0000","last_peered":"2024-04-25T17:08:12.176383+0000","last_clean":"2024-04-25T17:08:12.176383+0000","last_became_active":"2024-04-25T17:08:12.172786+0000","last_became_peered":"2024-04-25T17:08:12.172786+0000","last_unstale":"2024-04-25T17:08:12.176383+0000","last_undegraded":"2024-04-25T17:08:12.176383+0000","last_fullsized":"2024-04-25T17:08:12.176383+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.264530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495785+0000","last_change":"2024-04-25T17:08:12.158772+0000","last_active":"2024-04-25T17:08:13.495785+0000","last_peered":"2024-04-25T17:08:13.495785+0000","last_clean":"2024-04-25T17:08:13.495785+0000","last_became_active":"2024-04-25T17:08:12.158417+0000","last_became_peered":"2024-04-25T17:08:12.158417+0000","last_unstale":"2024-04-25T17:08:13.495785+0000","last_undegraded":"2024-04-25T17:08:13.495785+0000","last_fullsized":"2024-04-25T17:08:13.495785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:21.837887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163235+0000","last_change":"2024-04-25T17:08:12.162384+0000","last_active":"2024-04-25T17:08:12.163235+0000","last_peered":"2024-04-25T17:08:12.163235+0000","last_clean":"2024-04-25T17:08:12.163235+0000","last_became_active":"2024-04-25T17:08:12.161997+0000","last_became_peered":"2024-04-25T17:08:12.161997+0000","last_unstale":"2024-04-25T17:08:12.163235+0000","last_undegraded":"2024-04-25T17:08:12.163235+0000","last_fullsized":"2024-04-25T17:08:12.163235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:19.759938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.160211+0000","last_change":"2024-04-25T17:08:12.159015+0000","last_active":"2024-04-25T17:08:12.160211+0000","last_peered":"2024-04-25T17:08:12.160211+0000","last_clean":"2024-04-25T17:08:12.160211+0000","last_became_active":"2024-04-25T17:08:12.157999+0000","last_became_peered":"2024-04-25T17:08:12.157999+0000","last_unstale":"2024-04-25T17:08:12.160211+0000","last_undegraded":"2024-04-25T17:08:12.160211+0000","last_fullsized":"2024-04-25T17:08:12.160211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:15.456131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280965,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":43629,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":371473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874258,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":981294,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":802816,"data_stored":381132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":819200,"data_stored":389724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":864256,"data_stored":980070,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":798720,"data_stored":380550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":811008,"data_stored":382162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":815104,"data_stored":389210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":410049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":371473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":802816,"data_stored":380572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":835584,"data_stored":961841,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38227,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T17:08:18.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 -- 172.21.15.96:0/2487250285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc1050c0 msgr2=0x7fbafc1054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 --2- 172.21.15.96:0/2487250285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc1050c0 0x7fbafc1054d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009900 tx=0x7fbaf002a410 comp rx=0 tx=0).stop 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 -- 172.21.15.96:0/2487250285 shutdown_connections 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 --2- 172.21.15.96:0/2487250285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbafc102270 0x7fbafc10bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 --2- 172.21.15.96:0/2487250285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbafc101770 0x7fbafc101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 --2- 172.21.15.96:0/2487250285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc1050c0 0x7fbafc1054d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 -- 172.21.15.96:0/2487250285 >> 172.21.15.96:0/2487250285 conn(0x7fbafc0fd390 msgr2=0x7fbafc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 -- 172.21.15.96:0/2487250285 shutdown_connections 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.713+0000 7fbb02e05640 1 -- 172.21.15.96:0/2487250285 wait complete. 2024-04-25T17:08:18.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 Processor -- start 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 -- start start 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbafc101770 0x7fbafc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc102270 0x7fbafc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbafc1050c0 0x7fbafc1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc10f610 con 0x7fbafc102270 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc10f490 con 0x7fbafc101770 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb02e05640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc10f790 con 0x7fbafc1050c0 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbafbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc102270 0x7fbafc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbafbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc102270 0x7fbafc19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47652/0 (socket says 172.21.15.96:47652) 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbafbfff640 1 -- 172.21.15.96:0/3051703121 learned_addr learned my addr 172.21.15.96:0/3051703121 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb00ba1640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbafc101770 0x7fbafc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.714+0000 7fbb013a2640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbafc1050c0 0x7fbafc1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbafbfff640 1 -- 172.21.15.96:0/3051703121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbafc1050c0 msgr2=0x7fbafc1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbafbfff640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbafc1050c0 0x7fbafc1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbafbfff640 1 -- 172.21.15.96:0/3051703121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbafc101770 msgr2=0x7fbafc19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbafbfff640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbafc101770 0x7fbafc19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbafbfff640 1 -- 172.21.15.96:0/3051703121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc1a1ca0 con 0x7fbafc102270 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbafbfff640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc102270 0x7fbafc19a750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c920 tx=0x7fbae800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbaf9ffb640 1 -- 172.21.15.96:0/3051703121 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8007c70 con 0x7fbafc102270 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbaf9ffb640 1 -- 172.21.15.96:0/3051703121 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbae8007e10 con 0x7fbafc102270 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbaf9ffb640 1 -- 172.21.15.96:0/3051703121 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8018820 con 0x7fbafc102270 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc1a1f90 con 0x7fbafc102270 2024-04-25T17:08:18.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.715+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc1a23a0 con 0x7fbafc102270 2024-04-25T17:08:18.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.716+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc108a50 con 0x7fbafc102270 2024-04-25T17:08:18.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.717+0000 7fbaf9ffb640 1 -- 172.21.15.96:0/3051703121 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fbae800f450 con 0x7fbafc102270 2024-04-25T17:08:18.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.718+0000 7fbaf9ffb640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbaec0afb80 0x7fbaec0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:18.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.718+0000 7fbb00ba1640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbaec0afb80 0x7fbaec0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:18.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.718+0000 7fbaf9ffb640 1 -- 172.21.15.96:0/3051703121 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fbae80deca0 con 0x7fbafc102270 2024-04-25T17:08:18.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.719+0000 7fbb00ba1640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbaec0afb80 0x7fbaec0b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005ab0 tx=0x7fbaf0031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.723+0000 7fbaf9ffb640 1 -- 172.21.15.96:0/3051703121 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbae80e3070 con 0x7fbafc102270 2024-04-25T17:08:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:18 smithi097 ceph-mon[32602]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:18 smithi097 ceph-mon[32602]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T17:08:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:18 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2344384353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:18.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:18 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3819931771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:18.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:18 smithi171 ceph-mon[33189]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:08:18.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:18 smithi171 ceph-mon[33189]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T17:08:18.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:18 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2344384353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:08:18.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:18 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3819931771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:18.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.896+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbafc062fe0 con 0x7fbaec0afb80 2024-04-25T17:08:18.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.903+0000 7fbaf9ffb640 1 -- 172.21.15.96:0/3051703121 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258464 (secure 0 0 0) 0x7fbafc062fe0 con 0x7fbaec0afb80 2024-04-25T17:08:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:18.907 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T17:08:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbaec0afb80 msgr2=0x7fbaec0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbaec0afb80 0x7fbaec0b2090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005ab0 tx=0x7fbaf0031400 comp rx=0 tx=0).stop 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc102270 msgr2=0x7fbafc19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc102270 0x7fbafc19a750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c920 tx=0x7fbae800cde0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 shutdown_connections 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbaec0afb80 0x7fbaec0b2090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbafc1050c0 0x7fbafc1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbafc102270 0x7fbafc19a750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 --2- 172.21.15.96:0/3051703121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbafc101770 0x7fbafc19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 >> 172.21.15.96:0/3051703121 conn(0x7fbafc0fd390 msgr2=0x7fbafc105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 shutdown_connections 2024-04-25T17:08:18.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:18.906+0000 7fbb02e05640 1 -- 172.21.15.96:0/3051703121 wait complete. 2024-04-25T17:08:18.917 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-04-25T17:08:17.049972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":85,"num_read_kb":73,"num_write":162,"num_write_kb":2162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358832,"kb_used_data":9620,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582992,"statfs":{"total":1151940427776,"available":1151572983808,"internally_reserved":0,"allocated":9850880,"data_stored":6379550,"data_compressed":107190,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001410"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167139+0000","last_change":"2024-04-25T17:08:12.162430+0000","last_active":"2024-04-25T17:08:12.167139+0000","last_peered":"2024-04-25T17:08:12.167139+0000","last_clean":"2024-04-25T17:08:12.167139+0000","last_became_active":"2024-04-25T17:08:12.162052+0000","last_became_peered":"2024-04-25T17:08:12.162052+0000","last_unstale":"2024-04-25T17:08:12.167139+0000","last_undegraded":"2024-04-25T17:08:12.167139+0000","last_fullsized":"2024-04-25T17:08:12.167139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:14.457325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421957+0000","last_change":"2024-04-25T17:08:12.169139+0000","last_active":"2024-04-25T17:08:13.421957+0000","last_peered":"2024-04-25T17:08:13.421957+0000","last_clean":"2024-04-25T17:08:13.421957+0000","last_became_active":"2024-04-25T17:08:12.168715+0000","last_became_peered":"2024-04-25T17:08:12.168715+0000","last_unstale":"2024-04-25T17:08:13.421957+0000","last_undegraded":"2024-04-25T17:08:13.421957+0000","last_fullsized":"2024-04-25T17:08:13.421957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:57.357750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421733+0000","last_change":"2024-04-25T17:08:12.159141+0000","last_active":"2024-04-25T17:08:13.421733+0000","last_peered":"2024-04-25T17:08:13.421733+0000","last_clean":"2024-04-25T17:08:13.421733+0000","last_became_active":"2024-04-25T17:08:12.158796+0000","last_became_peered":"2024-04-25T17:08:12.158796+0000","last_unstale":"2024-04-25T17:08:13.421733+0000","last_undegraded":"2024-04-25T17:08:13.421733+0000","last_fullsized":"2024-04-25T17:08:13.421733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:55.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.161521+0000","last_change":"2024-04-25T17:08:12.160464+0000","last_active":"2024-04-25T17:08:12.161521+0000","last_peered":"2024-04-25T17:08:12.161521+0000","last_clean":"2024-04-25T17:08:12.161521+0000","last_became_active":"2024-04-25T17:08:12.160118+0000","last_became_peered":"2024-04-25T17:08:12.160118+0000","last_unstale":"2024-04-25T17:08:12.161521+0000","last_undegraded":"2024-04-25T17:08:12.161521+0000","last_fullsized":"2024-04-25T17:08:12.161521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:57.579423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495515+0000","last_change":"2024-04-25T17:08:12.174591+0000","last_active":"2024-04-25T17:08:13.495515+0000","last_peered":"2024-04-25T17:08:13.495515+0000","last_clean":"2024-04-25T17:08:13.495515+0000","last_became_active":"2024-04-25T17:08:12.174173+0000","last_became_peered":"2024-04-25T17:08:12.174173+0000","last_unstale":"2024-04-25T17:08:13.495515+0000","last_undegraded":"2024-04-25T17:08:13.495515+0000","last_fullsized":"2024-04-25T17:08:13.495515+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:12.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141095+0000","last_change":"2024-04-25T17:08:12.164230+0000","last_active":"2024-04-25T17:08:13.141095+0000","last_peered":"2024-04-25T17:08:13.141095+0000","last_clean":"2024-04-25T17:08:13.141095+0000","last_became_active":"2024-04-25T17:08:12.163868+0000","last_became_peered":"2024-04-25T17:08:12.163868+0000","last_unstale":"2024-04-25T17:08:13.141095+0000","last_undegraded":"2024-04-25T17:08:13.141095+0000","last_fullsized":"2024-04-25T17:08:13.141095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:12.604164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121611+0000","last_change":"2024-04-25T17:08:12.161627+0000","last_active":"2024-04-25T17:08:14.121611+0000","last_peered":"2024-04-25T17:08:14.121611+0000","last_clean":"2024-04-25T17:08:14.121611+0000","last_became_active":"2024-04-25T17:08:12.161146+0000","last_became_peered":"2024-04-25T17:08:12.161146+0000","last_unstale":"2024-04-25T17:08:14.121611+0000","last_undegraded":"2024-04-25T17:08:14.121611+0000","last_fullsized":"2024-04-25T17:08:14.121611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.972053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.166908+0000","last_change":"2024-04-25T17:08:12.165512+0000","last_active":"2024-04-25T17:08:12.166908+0000","last_peered":"2024-04-25T17:08:12.166908+0000","last_clean":"2024-04-25T17:08:12.166908+0000","last_became_active":"2024-04-25T17:08:12.164018+0000","last_became_peered":"2024-04-25T17:08:12.164018+0000","last_unstale":"2024-04-25T17:08:12.166908+0000","last_undegraded":"2024-04-25T17:08:12.166908+0000","last_fullsized":"2024-04-25T17:08:12.166908+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:45:29.650067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495771+0000","last_change":"2024-04-25T17:08:12.173313+0000","last_active":"2024-04-25T17:08:13.495771+0000","last_peered":"2024-04-25T17:08:13.495771+0000","last_clean":"2024-04-25T17:08:13.495771+0000","last_became_active":"2024-04-25T17:08:12.171269+0000","last_became_peered":"2024-04-25T17:08:12.171269+0000","last_unstale":"2024-04-25T17:08:13.495771+0000","last_undegraded":"2024-04-25T17:08:13.495771+0000","last_fullsized":"2024-04-25T17:08:13.495771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:39.565973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141559+0000","last_change":"2024-04-25T17:08:12.166114+0000","last_active":"2024-04-25T17:08:13.141559+0000","last_peered":"2024-04-25T17:08:13.141559+0000","last_clean":"2024-04-25T17:08:13.141559+0000","last_became_active":"2024-04-25T17:08:12.165737+0000","last_became_peered":"2024-04-25T17:08:12.165737+0000","last_unstale":"2024-04-25T17:08:13.141559+0000","last_undegraded":"2024-04-25T17:08:13.141559+0000","last_fullsized":"2024-04-25T17:08:13.141559+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:55.624239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122116+0000","last_change":"2024-04-25T17:08:12.160435+0000","last_active":"2024-04-25T17:08:14.122116+0000","last_peered":"2024-04-25T17:08:14.122116+0000","last_clean":"2024-04-25T17:08:14.122116+0000","last_became_active":"2024-04-25T17:08:12.160085+0000","last_became_peered":"2024-04-25T17:08:12.160085+0000","last_unstale":"2024-04-25T17:08:14.122116+0000","last_undegraded":"2024-04-25T17:08:14.122116+0000","last_fullsized":"2024-04-25T17:08:14.122116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.709567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174761+0000","last_change":"2024-04-25T17:08:12.173990+0000","last_active":"2024-04-25T17:08:12.174761+0000","last_peered":"2024-04-25T17:08:12.174761+0000","last_clean":"2024-04-25T17:08:12.174761+0000","last_became_active":"2024-04-25T17:08:12.172838+0000","last_became_peered":"2024-04-25T17:08:12.172838+0000","last_unstale":"2024-04-25T17:08:12.174761+0000","last_undegraded":"2024-04-25T17:08:12.174761+0000","last_fullsized":"2024-04-25T17:08:12.174761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:16.784335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.169084+0000","last_change":"2024-04-25T17:08:12.167579+0000","last_active":"2024-04-25T17:08:12.169084+0000","last_peered":"2024-04-25T17:08:12.169084+0000","last_clean":"2024-04-25T17:08:12.169084+0000","last_became_active":"2024-04-25T17:08:12.166583+0000","last_became_peered":"2024-04-25T17:08:12.166583+0000","last_unstale":"2024-04-25T17:08:12.169084+0000","last_undegraded":"2024-04-25T17:08:12.169084+0000","last_fullsized":"2024-04-25T17:08:12.169084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:55.855450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421592+0000","last_change":"2024-04-25T17:08:12.161263+0000","last_active":"2024-04-25T17:08:13.421592+0000","last_peered":"2024-04-25T17:08:13.421592+0000","last_clean":"2024-04-25T17:08:13.421592+0000","last_became_active":"2024-04-25T17:08:12.160078+0000","last_became_peered":"2024-04-25T17:08:12.160078+0000","last_unstale":"2024-04-25T17:08:13.421592+0000","last_undegraded":"2024-04-25T17:08:13.421592+0000","last_fullsized":"2024-04-25T17:08:13.421592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:56.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421594+0000","last_change":"2024-04-25T17:08:12.167318+0000","last_active":"2024-04-25T17:08:13.421594+0000","last_peered":"2024-04-25T17:08:13.421594+0000","last_clean":"2024-04-25T17:08:13.421594+0000","last_became_active":"2024-04-25T17:08:12.166964+0000","last_became_peered":"2024-04-25T17:08:12.166964+0000","last_unstale":"2024-04-25T17:08:13.421594+0000","last_undegraded":"2024-04-25T17:08:13.421594+0000","last_fullsized":"2024-04-25T17:08:13.421594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:25.967818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494787+0000","last_change":"2024-04-25T17:08:12.166417+0000","last_active":"2024-04-25T17:08:13.494787+0000","last_peered":"2024-04-25T17:08:13.494787+0000","last_clean":"2024-04-25T17:08:13.494787+0000","last_became_active":"2024-04-25T17:08:12.165928+0000","last_became_peered":"2024-04-25T17:08:12.165928+0000","last_unstale":"2024-04-25T17:08:13.494787+0000","last_undegraded":"2024-04-25T17:08:13.494787+0000","last_fullsized":"2024-04-25T17:08:13.494787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:24:22.359814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167674+0000","last_change":"2024-04-25T17:08:12.164952+0000","last_active":"2024-04-25T17:08:12.167674+0000","last_peered":"2024-04-25T17:08:12.167674+0000","last_clean":"2024-04-25T17:08:12.167674+0000","last_became_active":"2024-04-25T17:08:12.164595+0000","last_became_peered":"2024-04-25T17:08:12.164595+0000","last_unstale":"2024-04-25T17:08:12.167674+0000","last_undegraded":"2024-04-25T17:08:12.167674+0000","last_fullsized":"2024-04-25T17:08:12.167674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:48.456115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.154737+0000","last_change":"2024-04-25T17:08:12.153583+0000","last_active":"2024-04-25T17:08:12.154737+0000","last_peered":"2024-04-25T17:08:12.154737+0000","last_clean":"2024-04-25T17:08:12.154737+0000","last_became_active":"2024-04-25T17:08:12.153070+0000","last_became_peered":"2024-04-25T17:08:12.153070+0000","last_unstale":"2024-04-25T17:08:12.154737+0000","last_undegraded":"2024-04-25T17:08:12.154737+0000","last_fullsized":"2024-04-25T17:08:12.154737+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352707+0000","last_change":"2024-04-25T17:08:12.166837+0000","last_active":"2024-04-25T17:08:13.352707+0000","last_peered":"2024-04-25T17:08:13.352707+0000","last_clean":"2024-04-25T17:08:13.352707+0000","last_became_active":"2024-04-25T17:08:12.166477+0000","last_became_peered":"2024-04-25T17:08:12.166477+0000","last_unstale":"2024-04-25T17:08:13.352707+0000","last_undegraded":"2024-04-25T17:08:13.352707+0000","last_fullsized":"2024-04-25T17:08:13.352707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:50.492333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.170659+0000","last_change":"2024-04-25T17:08:11.153164+0000","last_active":"2024-04-25T17:08:12.170659+0000","last_peered":"2024-04-25T17:08:12.170659+0000","last_clean":"2024-04-25T17:08:12.170659+0000","last_became_active":"2024-04-25T17:08:11.152189+0000","last_became_peered":"2024-04-25T17:08:11.152189+0000","last_unstale":"2024-04-25T17:08:12.170659+0000","last_undegraded":"2024-04-25T17:08:12.170659+0000","last_fullsized":"2024-04-25T17:08:12.170659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:30.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.178622+0000","last_change":"2024-04-25T17:08:12.178117+0000","last_active":"2024-04-25T17:08:12.178622+0000","last_peered":"2024-04-25T17:08:12.178622+0000","last_clean":"2024-04-25T17:08:12.178622+0000","last_became_active":"2024-04-25T17:08:12.177767+0000","last_became_peered":"2024-04-25T17:08:12.177767+0000","last_unstale":"2024-04-25T17:08:12.178622+0000","last_undegraded":"2024-04-25T17:08:12.178622+0000","last_fullsized":"2024-04-25T17:08:12.178622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:41.464565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.179579+0000","last_change":"2024-04-25T17:08:11.153996+0000","last_active":"2024-04-25T17:08:12.179579+0000","last_peered":"2024-04-25T17:08:12.179579+0000","last_clean":"2024-04-25T17:08:12.179579+0000","last_became_active":"2024-04-25T17:08:11.152993+0000","last_became_peered":"2024-04-25T17:08:11.152993+0000","last_unstale":"2024-04-25T17:08:12.179579+0000","last_undegraded":"2024-04-25T17:08:12.179579+0000","last_fullsized":"2024-04-25T17:08:12.179579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:33.116616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422141+0000","last_change":"2024-04-25T17:08:12.163707+0000","last_active":"2024-04-25T17:08:13.422141+0000","last_peered":"2024-04-25T17:08:13.422141+0000","last_clean":"2024-04-25T17:08:13.422141+0000","last_became_active":"2024-04-25T17:08:12.161868+0000","last_became_peered":"2024-04-25T17:08:12.161868+0000","last_unstale":"2024-04-25T17:08:13.422141+0000","last_undegraded":"2024-04-25T17:08:13.422141+0000","last_fullsized":"2024-04-25T17:08:13.422141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.333036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496262+0000","last_change":"2024-04-25T17:08:11.158101+0000","last_active":"2024-04-25T17:08:13.496262+0000","last_peered":"2024-04-25T17:08:13.496262+0000","last_clean":"2024-04-25T17:08:13.496262+0000","last_became_active":"2024-04-25T17:08:11.157747+0000","last_became_peered":"2024-04-25T17:08:11.157747+0000","last_unstale":"2024-04-25T17:08:13.496262+0000","last_undegraded":"2024-04-25T17:08:13.496262+0000","last_fullsized":"2024-04-25T17:08:13.496262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:03.840823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495644+0000","last_change":"2024-04-25T17:08:12.170278+0000","last_active":"2024-04-25T17:08:13.495644+0000","last_peered":"2024-04-25T17:08:13.495644+0000","last_clean":"2024-04-25T17:08:13.495644+0000","last_became_active":"2024-04-25T17:08:12.169219+0000","last_became_peered":"2024-04-25T17:08:12.169219+0000","last_unstale":"2024-04-25T17:08:13.495644+0000","last_undegraded":"2024-04-25T17:08:13.495644+0000","last_fullsized":"2024-04-25T17:08:13.495644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:35.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496183+0000","last_change":"2024-04-25T17:08:11.153167+0000","last_active":"2024-04-25T17:08:13.496183+0000","last_peered":"2024-04-25T17:08:13.496183+0000","last_clean":"2024-04-25T17:08:13.496183+0000","last_became_active":"2024-04-25T17:08:11.152772+0000","last_became_peered":"2024-04-25T17:08:11.152772+0000","last_unstale":"2024-04-25T17:08:13.496183+0000","last_undegraded":"2024-04-25T17:08:13.496183+0000","last_fullsized":"2024-04-25T17:08:13.496183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:57.198963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.146203+0000","last_change":"2024-04-25T17:08:11.146203+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.146203+0000","last_undegraded":"2024-04-25T17:08:11.146203+0000","last_fullsized":"2024-04-25T17:08:11.146203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.159382+0000","last_change":"2024-04-25T17:08:11.158564+0000","last_active":"2024-04-25T17:08:11.159382+0000","last_peered":"2024-04-25T17:08:11.159382+0000","last_clean":"2024-04-25T17:08:11.159382+0000","last_became_active":"2024-04-25T17:08:11.158143+0000","last_became_peered":"2024-04-25T17:08:11.158143+0000","last_unstale":"2024-04-25T17:08:11.159382+0000","last_undegraded":"2024-04-25T17:08:11.159382+0000","last_fullsized":"2024-04-25T17:08:11.159382+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:44.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495878+0000","last_change":"2024-04-25T17:08:12.167584+0000","last_active":"2024-04-25T17:08:13.495878+0000","last_peered":"2024-04-25T17:08:13.495878+0000","last_clean":"2024-04-25T17:08:13.495878+0000","last_became_active":"2024-04-25T17:08:12.166717+0000","last_became_peered":"2024-04-25T17:08:12.166717+0000","last_unstale":"2024-04-25T17:08:13.495878+0000","last_undegraded":"2024-04-25T17:08:13.495878+0000","last_fullsized":"2024-04-25T17:08:13.495878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:38.047671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141009+0000","last_change":"2024-04-25T17:08:11.159254+0000","last_active":"2024-04-25T17:08:13.141009+0000","last_peered":"2024-04-25T17:08:13.141009+0000","last_clean":"2024-04-25T17:08:13.141009+0000","last_became_active":"2024-04-25T17:08:11.158864+0000","last_became_peered":"2024-04-25T17:08:11.158864+0000","last_unstale":"2024-04-25T17:08:13.141009+0000","last_undegraded":"2024-04-25T17:08:13.141009+0000","last_fullsized":"2024-04-25T17:08:13.141009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:17.795010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.144854+0000","last_change":"2024-04-25T17:08:11.144854+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.144854+0000","last_undegraded":"2024-04-25T17:08:11.144854+0000","last_fullsized":"2024-04-25T17:08:11.144854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.148562+0000","last_change":"2024-04-25T17:08:11.150980+0000","last_active":"2024-04-25T17:08:12.148562+0000","last_peered":"2024-04-25T17:08:12.148562+0000","last_clean":"2024-04-25T17:08:12.148562+0000","last_became_active":"2024-04-25T17:08:11.150580+0000","last_became_peered":"2024-04-25T17:08:11.150580+0000","last_unstale":"2024-04-25T17:08:12.148562+0000","last_undegraded":"2024-04-25T17:08:12.148562+0000","last_fullsized":"2024-04-25T17:08:12.148562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:59.539673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.175719+0000","last_change":"2024-04-25T17:08:12.171013+0000","last_active":"2024-04-25T17:08:12.175719+0000","last_peered":"2024-04-25T17:08:12.175719+0000","last_clean":"2024-04-25T17:08:12.175719+0000","last_became_active":"2024-04-25T17:08:12.169868+0000","last_became_peered":"2024-04-25T17:08:12.169868+0000","last_unstale":"2024-04-25T17:08:12.175719+0000","last_undegraded":"2024-04-25T17:08:12.175719+0000","last_fullsized":"2024-04-25T17:08:12.175719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:59.044063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.158357+0000","last_change":"2024-04-25T17:08:11.157618+0000","last_active":"2024-04-25T17:08:11.158357+0000","last_peered":"2024-04-25T17:08:11.158357+0000","last_clean":"2024-04-25T17:08:11.158357+0000","last_became_active":"2024-04-25T17:08:11.157052+0000","last_became_peered":"2024-04-25T17:08:11.157052+0000","last_unstale":"2024-04-25T17:08:11.158357+0000","last_undegraded":"2024-04-25T17:08:11.158357+0000","last_fullsized":"2024-04-25T17:08:11.158357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:09.902702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.151617+0000","last_change":"2024-04-25T17:08:12.151093+0000","last_active":"2024-04-25T17:08:12.151617+0000","last_peered":"2024-04-25T17:08:12.151617+0000","last_clean":"2024-04-25T17:08:12.151617+0000","last_became_active":"2024-04-25T17:08:12.150699+0000","last_became_peered":"2024-04-25T17:08:12.150699+0000","last_unstale":"2024-04-25T17:08:12.151617+0000","last_undegraded":"2024-04-25T17:08:12.151617+0000","last_fullsized":"2024-04-25T17:08:12.151617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:46.331493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141389+0000","last_change":"2024-04-25T17:08:11.157673+0000","last_active":"2024-04-25T17:08:13.141389+0000","last_peered":"2024-04-25T17:08:13.141389+0000","last_clean":"2024-04-25T17:08:13.141389+0000","last_became_active":"2024-04-25T17:08:11.157089+0000","last_became_peered":"2024-04-25T17:08:11.157089+0000","last_unstale":"2024-04-25T17:08:13.141389+0000","last_undegraded":"2024-04-25T17:08:13.141389+0000","last_fullsized":"2024-04-25T17:08:13.141389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:47.586003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495779+0000","last_change":"2024-04-25T17:08:12.170666+0000","last_active":"2024-04-25T17:08:13.495779+0000","last_peered":"2024-04-25T17:08:13.495779+0000","last_clean":"2024-04-25T17:08:13.495779+0000","last_became_active":"2024-04-25T17:08:12.169607+0000","last_became_peered":"2024-04-25T17:08:12.169607+0000","last_unstale":"2024-04-25T17:08:13.495779+0000","last_undegraded":"2024-04-25T17:08:13.495779+0000","last_fullsized":"2024-04-25T17:08:13.495779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121887+0000","last_change":"2024-04-25T17:08:11.153407+0000","last_active":"2024-04-25T17:08:14.121887+0000","last_peered":"2024-04-25T17:08:14.121887+0000","last_clean":"2024-04-25T17:08:14.121887+0000","last_became_active":"2024-04-25T17:08:11.152932+0000","last_became_peered":"2024-04-25T17:08:11.152932+0000","last_unstale":"2024-04-25T17:08:14.121887+0000","last_undegraded":"2024-04-25T17:08:14.121887+0000","last_fullsized":"2024-04-25T17:08:14.121887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:06.712620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422086+0000","last_change":"2024-04-25T17:08:12.173788+0000","last_active":"2024-04-25T17:08:13.422086+0000","last_peered":"2024-04-25T17:08:13.422086+0000","last_clean":"2024-04-25T17:08:13.422086+0000","last_became_active":"2024-04-25T17:08:12.172714+0000","last_became_peered":"2024-04-25T17:08:12.172714+0000","last_unstale":"2024-04-25T17:08:13.422086+0000","last_undegraded":"2024-04-25T17:08:13.422086+0000","last_fullsized":"2024-04-25T17:08:13.422086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:18.892187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.161140+0000","last_change":"2024-04-25T17:08:11.159802+0000","last_active":"2024-04-25T17:08:11.161140+0000","last_peered":"2024-04-25T17:08:11.161140+0000","last_clean":"2024-04-25T17:08:11.161140+0000","last_became_active":"2024-04-25T17:08:11.159431+0000","last_became_peered":"2024-04-25T17:08:11.159431+0000","last_unstale":"2024-04-25T17:08:11.161140+0000","last_undegraded":"2024-04-25T17:08:11.161140+0000","last_fullsized":"2024-04-25T17:08:11.161140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:45.961985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495822+0000","last_change":"2024-04-25T17:08:12.162703+0000","last_active":"2024-04-25T17:08:13.495822+0000","last_peered":"2024-04-25T17:08:13.495822+0000","last_clean":"2024-04-25T17:08:13.495822+0000","last_became_active":"2024-04-25T17:08:12.162361+0000","last_became_peered":"2024-04-25T17:08:12.162361+0000","last_unstale":"2024-04-25T17:08:13.495822+0000","last_undegraded":"2024-04-25T17:08:13.495822+0000","last_fullsized":"2024-04-25T17:08:13.495822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:51.262319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'6","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141260+0000","last_change":"2024-04-25T17:08:11.153938+0000","last_active":"2024-04-25T17:08:13.141260+0000","last_peered":"2024-04-25T17:08:13.141260+0000","last_clean":"2024-04-25T17:08:13.141260+0000","last_became_active":"2024-04-25T17:08:11.153591+0000","last_became_peered":"2024-04-25T17:08:11.153591+0000","last_unstale":"2024-04-25T17:08:13.141260+0000","last_undegraded":"2024-04-25T17:08:13.141260+0000","last_fullsized":"2024-04-25T17:08:13.141260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496204+0000","last_change":"2024-04-25T17:08:12.156045+0000","last_active":"2024-04-25T17:08:13.496204+0000","last_peered":"2024-04-25T17:08:13.496204+0000","last_clean":"2024-04-25T17:08:13.496204+0000","last_became_active":"2024-04-25T17:08:12.155657+0000","last_became_peered":"2024-04-25T17:08:12.155657+0000","last_unstale":"2024-04-25T17:08:13.496204+0000","last_undegraded":"2024-04-25T17:08:13.496204+0000","last_fullsized":"2024-04-25T17:08:13.496204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352496+0000","last_change":"2024-04-25T17:08:11.162009+0000","last_active":"2024-04-25T17:08:13.352496+0000","last_peered":"2024-04-25T17:08:13.352496+0000","last_clean":"2024-04-25T17:08:13.352496+0000","last_became_active":"2024-04-25T17:08:11.161574+0000","last_became_peered":"2024-04-25T17:08:11.161574+0000","last_unstale":"2024-04-25T17:08:13.352496+0000","last_undegraded":"2024-04-25T17:08:13.352496+0000","last_fullsized":"2024-04-25T17:08:13.352496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:16.760355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163159+0000","last_change":"2024-04-25T17:08:12.159522+0000","last_active":"2024-04-25T17:08:12.163159+0000","last_peered":"2024-04-25T17:08:12.163159+0000","last_clean":"2024-04-25T17:08:12.163159+0000","last_became_active":"2024-04-25T17:08:12.159036+0000","last_became_peered":"2024-04-25T17:08:12.159036+0000","last_unstale":"2024-04-25T17:08:12.163159+0000","last_undegraded":"2024-04-25T17:08:12.163159+0000","last_fullsized":"2024-04-25T17:08:12.163159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:18.534810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.162996+0000","last_change":"2024-04-25T17:08:11.162487+0000","last_active":"2024-04-25T17:08:11.162996+0000","last_peered":"2024-04-25T17:08:11.162996+0000","last_clean":"2024-04-25T17:08:11.162996+0000","last_became_active":"2024-04-25T17:08:11.161961+0000","last_became_peered":"2024-04-25T17:08:11.161961+0000","last_unstale":"2024-04-25T17:08:11.162996+0000","last_undegraded":"2024-04-25T17:08:11.162996+0000","last_fullsized":"2024-04-25T17:08:11.162996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:09.095387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121943+0000","last_change":"2024-04-25T17:08:12.164107+0000","last_active":"2024-04-25T17:08:14.121943+0000","last_peered":"2024-04-25T17:08:14.121943+0000","last_clean":"2024-04-25T17:08:14.121943+0000","last_became_active":"2024-04-25T17:08:12.163741+0000","last_became_peered":"2024-04-25T17:08:12.163741+0000","last_unstale":"2024-04-25T17:08:14.121943+0000","last_undegraded":"2024-04-25T17:08:14.121943+0000","last_fullsized":"2024-04-25T17:08:14.121943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.552833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.146579+0000","last_change":"2024-04-25T17:08:11.153383+0000","last_active":"2024-04-25T17:08:12.146579+0000","last_peered":"2024-04-25T17:08:12.146579+0000","last_clean":"2024-04-25T17:08:12.146579+0000","last_became_active":"2024-04-25T17:08:11.152573+0000","last_became_peered":"2024-04-25T17:08:11.152573+0000","last_unstale":"2024-04-25T17:08:12.146579+0000","last_undegraded":"2024-04-25T17:08:12.146579+0000","last_fullsized":"2024-04-25T17:08:12.146579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:13.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121535+0000","last_change":"2024-04-25T17:08:12.165939+0000","last_active":"2024-04-25T17:08:14.121535+0000","last_peered":"2024-04-25T17:08:14.121535+0000","last_clean":"2024-04-25T17:08:14.121535+0000","last_became_active":"2024-04-25T17:08:12.165569+0000","last_became_peered":"2024-04-25T17:08:12.165569+0000","last_unstale":"2024-04-25T17:08:14.121535+0000","last_undegraded":"2024-04-25T17:08:14.121535+0000","last_fullsized":"2024-04-25T17:08:14.121535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:15:16.230457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495142+0000","last_change":"2024-04-25T17:08:11.158927+0000","last_active":"2024-04-25T17:08:13.495142+0000","last_peered":"2024-04-25T17:08:13.495142+0000","last_clean":"2024-04-25T17:08:13.495142+0000","last_became_active":"2024-04-25T17:08:11.158528+0000","last_became_peered":"2024-04-25T17:08:11.158528+0000","last_unstale":"2024-04-25T17:08:13.495142+0000","last_undegraded":"2024-04-25T17:08:13.495142+0000","last_fullsized":"2024-04-25T17:08:13.495142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.308627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.143338+0000","last_change":"2024-04-25T17:08:11.143338+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.143338+0000","last_undegraded":"2024-04-25T17:08:11.143338+0000","last_fullsized":"2024-04-25T17:08:11.143338+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495648+0000","last_change":"2024-04-25T17:08:11.153214+0000","last_active":"2024-04-25T17:08:13.495648+0000","last_peered":"2024-04-25T17:08:13.495648+0000","last_clean":"2024-04-25T17:08:13.495648+0000","last_became_active":"2024-04-25T17:08:11.152728+0000","last_became_peered":"2024-04-25T17:08:11.152728+0000","last_unstale":"2024-04-25T17:08:13.495648+0000","last_undegraded":"2024-04-25T17:08:13.495648+0000","last_fullsized":"2024-04-25T17:08:13.495648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.338549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167104+0000","last_change":"2024-04-25T17:08:12.162167+0000","last_active":"2024-04-25T17:08:12.167104+0000","last_peered":"2024-04-25T17:08:12.167104+0000","last_clean":"2024-04-25T17:08:12.167104+0000","last_became_active":"2024-04-25T17:08:12.160896+0000","last_became_peered":"2024-04-25T17:08:12.160896+0000","last_unstale":"2024-04-25T17:08:12.167104+0000","last_undegraded":"2024-04-25T17:08:12.167104+0000","last_fullsized":"2024-04-25T17:08:12.167104+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:20.437234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.141261+0000","last_change":"2024-04-25T17:08:11.158995+0000","last_active":"2024-04-25T17:08:12.141261+0000","last_peered":"2024-04-25T17:08:12.141261+0000","last_clean":"2024-04-25T17:08:12.141261+0000","last_became_active":"2024-04-25T17:08:11.158644+0000","last_became_peered":"2024-04-25T17:08:11.158644+0000","last_unstale":"2024-04-25T17:08:12.141261+0000","last_undegraded":"2024-04-25T17:08:12.141261+0000","last_fullsized":"2024-04-25T17:08:12.141261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:22.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141392+0000","last_change":"2024-04-25T17:08:12.161635+0000","last_active":"2024-04-25T17:08:13.141392+0000","last_peered":"2024-04-25T17:08:13.141392+0000","last_clean":"2024-04-25T17:08:13.141392+0000","last_became_active":"2024-04-25T17:08:12.153728+0000","last_became_peered":"2024-04-25T17:08:12.153728+0000","last_unstale":"2024-04-25T17:08:13.141392+0000","last_undegraded":"2024-04-25T17:08:13.141392+0000","last_fullsized":"2024-04-25T17:08:13.141392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:02.003550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121630+0000","last_change":"2024-04-25T17:08:11.150490+0000","last_active":"2024-04-25T17:08:14.121630+0000","last_peered":"2024-04-25T17:08:14.121630+0000","last_clean":"2024-04-25T17:08:14.121630+0000","last_became_active":"2024-04-25T17:08:11.148391+0000","last_became_peered":"2024-04-25T17:08:11.148391+0000","last_unstale":"2024-04-25T17:08:14.121630+0000","last_undegraded":"2024-04-25T17:08:14.121630+0000","last_fullsized":"2024-04-25T17:08:14.121630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:40.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141295+0000","last_change":"2024-04-25T17:08:12.168136+0000","last_active":"2024-04-25T17:08:13.141295+0000","last_peered":"2024-04-25T17:08:13.141295+0000","last_clean":"2024-04-25T17:08:13.141295+0000","last_became_active":"2024-04-25T17:08:12.167786+0000","last_became_peered":"2024-04-25T17:08:12.167786+0000","last_unstale":"2024-04-25T17:08:13.141295+0000","last_undegraded":"2024-04-25T17:08:13.141295+0000","last_fullsized":"2024-04-25T17:08:13.141295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:22.869503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.149194+0000","last_change":"2024-04-25T17:08:11.149194+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.149194+0000","last_undegraded":"2024-04-25T17:08:11.149194+0000","last_fullsized":"2024-04-25T17:08:11.149194+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.149557+0000","last_change":"2024-04-25T17:08:11.149085+0000","last_active":"2024-04-25T17:08:12.149557+0000","last_peered":"2024-04-25T17:08:12.149557+0000","last_clean":"2024-04-25T17:08:12.149557+0000","last_became_active":"2024-04-25T17:08:11.148712+0000","last_became_peered":"2024-04-25T17:08:11.148712+0000","last_unstale":"2024-04-25T17:08:12.149557+0000","last_undegraded":"2024-04-25T17:08:12.149557+0000","last_fullsized":"2024-04-25T17:08:12.149557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:12.764034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.178626+0000","last_change":"2024-04-25T17:08:12.178154+0000","last_active":"2024-04-25T17:08:12.178626+0000","last_peered":"2024-04-25T17:08:12.178626+0000","last_clean":"2024-04-25T17:08:12.178626+0000","last_became_active":"2024-04-25T17:08:12.177773+0000","last_became_peered":"2024-04-25T17:08:12.177773+0000","last_unstale":"2024-04-25T17:08:12.178626+0000","last_undegraded":"2024-04-25T17:08:12.178626+0000","last_fullsized":"2024-04-25T17:08:12.178626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:30.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.148011+0000","last_change":"2024-04-25T17:08:11.152932+0000","last_active":"2024-04-25T17:08:12.148011+0000","last_peered":"2024-04-25T17:08:12.148011+0000","last_clean":"2024-04-25T17:08:12.148011+0000","last_became_active":"2024-04-25T17:08:11.152530+0000","last_became_peered":"2024-04-25T17:08:11.152530+0000","last_unstale":"2024-04-25T17:08:12.148011+0000","last_undegraded":"2024-04-25T17:08:12.148011+0000","last_fullsized":"2024-04-25T17:08:12.148011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121425+0000","last_change":"2024-04-25T17:06:55.951326+0000","last_active":"2024-04-25T17:08:14.121425+0000","last_peered":"2024-04-25T17:08:14.121425+0000","last_clean":"2024-04-25T17:08:14.121425+0000","last_became_active":"2024-04-25T17:06:55.946641+0000","last_became_peered":"2024-04-25T17:06:55.946641+0000","last_unstale":"2024-04-25T17:08:14.121425+0000","last_undegraded":"2024-04-25T17:08:14.121425+0000","last_fullsized":"2024-04-25T17:08:14.121425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_clean_scrub_stamp":"2024-04-25T17:03:27.394276+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:58.308843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163189+0000","last_change":"2024-04-25T17:08:12.162065+0000","last_active":"2024-04-25T17:08:12.163189+0000","last_peered":"2024-04-25T17:08:12.163189+0000","last_clean":"2024-04-25T17:08:12.163189+0000","last_became_active":"2024-04-25T17:08:12.161733+0000","last_became_peered":"2024-04-25T17:08:12.161733+0000","last_unstale":"2024-04-25T17:08:12.163189+0000","last_undegraded":"2024-04-25T17:08:12.163189+0000","last_fullsized":"2024-04-25T17:08:12.163189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:05.132252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121612+0000","last_change":"2024-04-25T17:08:11.155941+0000","last_active":"2024-04-25T17:08:14.121612+0000","last_peered":"2024-04-25T17:08:14.121612+0000","last_clean":"2024-04-25T17:08:14.121612+0000","last_became_active":"2024-04-25T17:08:11.155381+0000","last_became_peered":"2024-04-25T17:08:11.155381+0000","last_unstale":"2024-04-25T17:08:14.121612+0000","last_undegraded":"2024-04-25T17:08:14.121612+0000","last_fullsized":"2024-04-25T17:08:14.121612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:28.423462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496103+0000","last_change":"2024-04-25T17:08:11.158960+0000","last_active":"2024-04-25T17:08:13.496103+0000","last_peered":"2024-04-25T17:08:13.496103+0000","last_clean":"2024-04-25T17:08:13.496103+0000","last_became_active":"2024-04-25T17:08:11.158609+0000","last_became_peered":"2024-04-25T17:08:11.158609+0000","last_unstale":"2024-04-25T17:08:13.496103+0000","last_undegraded":"2024-04-25T17:08:13.496103+0000","last_fullsized":"2024-04-25T17:08:13.496103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:19.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163020+0000","last_change":"2024-04-25T17:08:12.161483+0000","last_active":"2024-04-25T17:08:12.163020+0000","last_peered":"2024-04-25T17:08:12.163020+0000","last_clean":"2024-04-25T17:08:12.163020+0000","last_became_active":"2024-04-25T17:08:12.158586+0000","last_became_peered":"2024-04-25T17:08:12.158586+0000","last_unstale":"2024-04-25T17:08:12.163020+0000","last_undegraded":"2024-04-25T17:08:12.163020+0000","last_fullsized":"2024-04-25T17:08:12.163020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.141542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.141400+0000","last_change":"2024-04-25T17:08:11.161497+0000","last_active":"2024-04-25T17:08:12.141400+0000","last_peered":"2024-04-25T17:08:12.141400+0000","last_clean":"2024-04-25T17:08:12.141400+0000","last_became_active":"2024-04-25T17:08:11.160990+0000","last_became_peered":"2024-04-25T17:08:11.160990+0000","last_unstale":"2024-04-25T17:08:12.141400+0000","last_undegraded":"2024-04-25T17:08:12.141400+0000","last_fullsized":"2024-04-25T17:08:12.141400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:59.682133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176351+0000","last_change":"2024-04-25T17:08:12.171214+0000","last_active":"2024-04-25T17:08:12.176351+0000","last_peered":"2024-04-25T17:08:12.176351+0000","last_clean":"2024-04-25T17:08:12.176351+0000","last_became_active":"2024-04-25T17:08:12.170877+0000","last_became_peered":"2024-04-25T17:08:12.170877+0000","last_unstale":"2024-04-25T17:08:12.176351+0000","last_undegraded":"2024-04-25T17:08:12.176351+0000","last_fullsized":"2024-04-25T17:08:12.176351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:18.267340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176348+0000","last_change":"2024-04-25T17:08:11.154819+0000","last_active":"2024-04-25T17:08:12.176348+0000","last_peered":"2024-04-25T17:08:12.176348+0000","last_clean":"2024-04-25T17:08:12.176348+0000","last_became_active":"2024-04-25T17:08:11.154006+0000","last_became_peered":"2024-04-25T17:08:11.154006+0000","last_unstale":"2024-04-25T17:08:12.176348+0000","last_undegraded":"2024-04-25T17:08:12.176348+0000","last_fullsized":"2024-04-25T17:08:12.176348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.063058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422056+0000","last_change":"2024-04-25T17:08:12.167301+0000","last_active":"2024-04-25T17:08:13.422056+0000","last_peered":"2024-04-25T17:08:13.422056+0000","last_clean":"2024-04-25T17:08:13.422056+0000","last_became_active":"2024-04-25T17:08:12.166944+0000","last_became_peered":"2024-04-25T17:08:12.166944+0000","last_unstale":"2024-04-25T17:08:13.422056+0000","last_undegraded":"2024-04-25T17:08:13.422056+0000","last_fullsized":"2024-04-25T17:08:13.422056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:46.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495108+0000","last_change":"2024-04-25T17:08:11.156616+0000","last_active":"2024-04-25T17:08:13.495108+0000","last_peered":"2024-04-25T17:08:13.495108+0000","last_clean":"2024-04-25T17:08:13.495108+0000","last_became_active":"2024-04-25T17:08:11.156256+0000","last_became_peered":"2024-04-25T17:08:11.156256+0000","last_unstale":"2024-04-25T17:08:13.495108+0000","last_undegraded":"2024-04-25T17:08:13.495108+0000","last_fullsized":"2024-04-25T17:08:13.495108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:37.539773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495319+0000","last_change":"2024-04-25T17:08:12.175216+0000","last_active":"2024-04-25T17:08:13.495319+0000","last_peered":"2024-04-25T17:08:13.495319+0000","last_clean":"2024-04-25T17:08:13.495319+0000","last_became_active":"2024-04-25T17:08:12.174521+0000","last_became_peered":"2024-04-25T17:08:12.174521+0000","last_unstale":"2024-04-25T17:08:13.495319+0000","last_undegraded":"2024-04-25T17:08:13.495319+0000","last_fullsized":"2024-04-25T17:08:13.495319+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.322733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.152221+0000","last_change":"2024-04-25T17:08:11.162287+0000","last_active":"2024-04-25T17:08:12.152221+0000","last_peered":"2024-04-25T17:08:12.152221+0000","last_clean":"2024-04-25T17:08:12.152221+0000","last_became_active":"2024-04-25T17:08:11.158297+0000","last_became_peered":"2024-04-25T17:08:11.158297+0000","last_unstale":"2024-04-25T17:08:12.152221+0000","last_undegraded":"2024-04-25T17:08:12.152221+0000","last_fullsized":"2024-04-25T17:08:12.152221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:15.758880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167929+0000","last_change":"2024-04-25T17:08:12.163535+0000","last_active":"2024-04-25T17:08:12.167929+0000","last_peered":"2024-04-25T17:08:12.167929+0000","last_clean":"2024-04-25T17:08:12.167929+0000","last_became_active":"2024-04-25T17:08:12.162848+0000","last_became_peered":"2024-04-25T17:08:12.162848+0000","last_unstale":"2024-04-25T17:08:12.167929+0000","last_undegraded":"2024-04-25T17:08:12.167929+0000","last_fullsized":"2024-04-25T17:08:12.167929+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:36.839617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352187+0000","last_change":"2024-04-25T17:08:11.162780+0000","last_active":"2024-04-25T17:08:13.352187+0000","last_peered":"2024-04-25T17:08:13.352187+0000","last_clean":"2024-04-25T17:08:13.352187+0000","last_became_active":"2024-04-25T17:08:11.162385+0000","last_became_peered":"2024-04-25T17:08:11.162385+0000","last_unstale":"2024-04-25T17:08:13.352187+0000","last_undegraded":"2024-04-25T17:08:13.352187+0000","last_fullsized":"2024-04-25T17:08:13.352187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:25.420740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174051+0000","last_change":"2024-04-25T17:08:12.170828+0000","last_active":"2024-04-25T17:08:12.174051+0000","last_peered":"2024-04-25T17:08:12.174051+0000","last_clean":"2024-04-25T17:08:12.174051+0000","last_became_active":"2024-04-25T17:08:12.170479+0000","last_became_peered":"2024-04-25T17:08:12.170479+0000","last_unstale":"2024-04-25T17:08:12.174051+0000","last_undegraded":"2024-04-25T17:08:12.174051+0000","last_fullsized":"2024-04-25T17:08:12.174051+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:30.418656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495662+0000","last_change":"2024-04-25T17:08:11.156754+0000","last_active":"2024-04-25T17:08:13.495662+0000","last_peered":"2024-04-25T17:08:13.495662+0000","last_clean":"2024-04-25T17:08:13.495662+0000","last_became_active":"2024-04-25T17:08:11.156414+0000","last_became_peered":"2024-04-25T17:08:11.156414+0000","last_unstale":"2024-04-25T17:08:13.495662+0000","last_undegraded":"2024-04-25T17:08:13.495662+0000","last_fullsized":"2024-04-25T17:08:13.495662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:10.364234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121978+0000","last_change":"2024-04-25T17:08:12.162032+0000","last_active":"2024-04-25T17:08:14.121978+0000","last_peered":"2024-04-25T17:08:14.121978+0000","last_clean":"2024-04-25T17:08:14.121978+0000","last_became_active":"2024-04-25T17:08:12.161017+0000","last_became_peered":"2024-04-25T17:08:12.161017+0000","last_unstale":"2024-04-25T17:08:14.121978+0000","last_undegraded":"2024-04-25T17:08:14.121978+0000","last_fullsized":"2024-04-25T17:08:14.121978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.472984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352660+0000","last_change":"2024-04-25T17:08:11.149550+0000","last_active":"2024-04-25T17:08:13.352660+0000","last_peered":"2024-04-25T17:08:13.352660+0000","last_clean":"2024-04-25T17:08:13.352660+0000","last_became_active":"2024-04-25T17:08:11.149082+0000","last_became_peered":"2024-04-25T17:08:11.149082+0000","last_unstale":"2024-04-25T17:08:13.352660+0000","last_undegraded":"2024-04-25T17:08:13.352660+0000","last_fullsized":"2024-04-25T17:08:13.352660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:14.565877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122122+0000","last_change":"2024-04-25T17:08:12.160331+0000","last_active":"2024-04-25T17:08:14.122122+0000","last_peered":"2024-04-25T17:08:14.122122+0000","last_clean":"2024-04-25T17:08:14.122122+0000","last_became_active":"2024-04-25T17:08:12.159931+0000","last_became_peered":"2024-04-25T17:08:12.159931+0000","last_unstale":"2024-04-25T17:08:14.122122+0000","last_undegraded":"2024-04-25T17:08:14.122122+0000","last_fullsized":"2024-04-25T17:08:14.122122+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:37.246974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T17:08:11.163558+0000","last_change":"2024-04-25T17:08:11.163118+0000","last_active":"2024-04-25T17:08:11.163558+0000","last_peered":"2024-04-25T17:08:11.163558+0000","last_clean":"2024-04-25T17:08:11.163558+0000","last_became_active":"2024-04-25T17:08:11.161691+0000","last_became_peered":"2024-04-25T17:08:11.161691+0000","last_unstale":"2024-04-25T17:08:11.163558+0000","last_undegraded":"2024-04-25T17:08:11.163558+0000","last_fullsized":"2024-04-25T17:08:11.163558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:23.135262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121927+0000","last_change":"2024-04-25T17:08:12.168387+0000","last_active":"2024-04-25T17:08:14.121927+0000","last_peered":"2024-04-25T17:08:14.121927+0000","last_clean":"2024-04-25T17:08:14.121927+0000","last_became_active":"2024-04-25T17:08:12.168008+0000","last_became_peered":"2024-04-25T17:08:12.168008+0000","last_unstale":"2024-04-25T17:08:14.121927+0000","last_undegraded":"2024-04-25T17:08:14.121927+0000","last_fullsized":"2024-04-25T17:08:14.121927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:08.438945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.174728+0000","last_change":"2024-04-25T17:08:11.149181+0000","last_active":"2024-04-25T17:08:12.174728+0000","last_peered":"2024-04-25T17:08:12.174728+0000","last_clean":"2024-04-25T17:08:12.174728+0000","last_became_active":"2024-04-25T17:08:11.148847+0000","last_became_peered":"2024-04-25T17:08:11.148847+0000","last_unstale":"2024-04-25T17:08:12.174728+0000","last_undegraded":"2024-04-25T17:08:12.174728+0000","last_fullsized":"2024-04-25T17:08:12.174728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:40.720578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122115+0000","last_change":"2024-04-25T17:08:12.162173+0000","last_active":"2024-04-25T17:08:14.122115+0000","last_peered":"2024-04-25T17:08:14.122115+0000","last_clean":"2024-04-25T17:08:14.122115+0000","last_became_active":"2024-04-25T17:08:12.161818+0000","last_became_peered":"2024-04-25T17:08:12.161818+0000","last_unstale":"2024-04-25T17:08:14.122115+0000","last_undegraded":"2024-04-25T17:08:14.122115+0000","last_fullsized":"2024-04-25T17:08:14.122115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:28.872937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.167954+0000","last_change":"2024-04-25T17:08:12.165542+0000","last_active":"2024-04-25T17:08:12.167954+0000","last_peered":"2024-04-25T17:08:12.167954+0000","last_clean":"2024-04-25T17:08:12.167954+0000","last_became_active":"2024-04-25T17:08:12.165190+0000","last_became_peered":"2024-04-25T17:08:12.165190+0000","last_unstale":"2024-04-25T17:08:12.167954+0000","last_undegraded":"2024-04-25T17:08:12.167954+0000","last_fullsized":"2024-04-25T17:08:12.167954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:24.414927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352355+0000","last_change":"2024-04-25T17:08:12.161429+0000","last_active":"2024-04-25T17:08:13.352355+0000","last_peered":"2024-04-25T17:08:13.352355+0000","last_clean":"2024-04-25T17:08:13.352355+0000","last_became_active":"2024-04-25T17:08:12.159184+0000","last_became_peered":"2024-04-25T17:08:12.159184+0000","last_unstale":"2024-04-25T17:08:13.352355+0000","last_undegraded":"2024-04-25T17:08:13.352355+0000","last_fullsized":"2024-04-25T17:08:13.352355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:56:21.366775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352298+0000","last_change":"2024-04-25T17:08:12.169948+0000","last_active":"2024-04-25T17:08:13.352298+0000","last_peered":"2024-04-25T17:08:13.352298+0000","last_clean":"2024-04-25T17:08:13.352298+0000","last_became_active":"2024-04-25T17:08:12.169592+0000","last_became_peered":"2024-04-25T17:08:12.169592+0000","last_unstale":"2024-04-25T17:08:13.352298+0000","last_undegraded":"2024-04-25T17:08:13.352298+0000","last_fullsized":"2024-04-25T17:08:13.352298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:22.749954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421852+0000","last_change":"2024-04-25T17:08:12.169785+0000","last_active":"2024-04-25T17:08:13.421852+0000","last_peered":"2024-04-25T17:08:13.421852+0000","last_clean":"2024-04-25T17:08:13.421852+0000","last_became_active":"2024-04-25T17:08:12.169226+0000","last_became_peered":"2024-04-25T17:08:12.169226+0000","last_unstale":"2024-04-25T17:08:13.421852+0000","last_undegraded":"2024-04-25T17:08:13.421852+0000","last_fullsized":"2024-04-25T17:08:13.421852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.232112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.162831+0000","last_change":"2024-04-25T17:08:12.157992+0000","last_active":"2024-04-25T17:08:12.162831+0000","last_peered":"2024-04-25T17:08:12.162831+0000","last_clean":"2024-04-25T17:08:12.162831+0000","last_became_active":"2024-04-25T17:08:12.157636+0000","last_became_peered":"2024-04-25T17:08:12.157636+0000","last_unstale":"2024-04-25T17:08:12.162831+0000","last_undegraded":"2024-04-25T17:08:12.162831+0000","last_fullsized":"2024-04-25T17:08:12.162831+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:25.138871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.162331+0000","last_change":"2024-04-25T17:08:12.160977+0000","last_active":"2024-04-25T17:08:12.162331+0000","last_peered":"2024-04-25T17:08:12.162331+0000","last_clean":"2024-04-25T17:08:12.162331+0000","last_became_active":"2024-04-25T17:08:12.160571+0000","last_became_peered":"2024-04-25T17:08:12.160571+0000","last_unstale":"2024-04-25T17:08:12.162331+0000","last_undegraded":"2024-04-25T17:08:12.162331+0000","last_fullsized":"2024-04-25T17:08:12.162331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:13:47.932677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352838+0000","last_change":"2024-04-25T17:08:12.158566+0000","last_active":"2024-04-25T17:08:13.352838+0000","last_peered":"2024-04-25T17:08:13.352838+0000","last_clean":"2024-04-25T17:08:13.352838+0000","last_became_active":"2024-04-25T17:08:12.157911+0000","last_became_peered":"2024-04-25T17:08:12.157911+0000","last_unstale":"2024-04-25T17:08:13.352838+0000","last_undegraded":"2024-04-25T17:08:13.352838+0000","last_fullsized":"2024-04-25T17:08:13.352838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:49.665779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176417+0000","last_change":"2024-04-25T17:08:12.174931+0000","last_active":"2024-04-25T17:08:12.176417+0000","last_peered":"2024-04-25T17:08:12.176417+0000","last_clean":"2024-04-25T17:08:12.176417+0000","last_became_active":"2024-04-25T17:08:12.174571+0000","last_became_peered":"2024-04-25T17:08:12.174571+0000","last_unstale":"2024-04-25T17:08:12.176417+0000","last_undegraded":"2024-04-25T17:08:12.176417+0000","last_fullsized":"2024-04-25T17:08:12.176417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:40.372717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-04-25T17:08:11.147694+0000","last_change":"2024-04-25T17:08:11.147694+0000","last_active":"2024-04-25T17:08:11.128828+0000","last_peered":"2024-04-25T17:08:11.128828+0000","last_clean":"2024-04-25T17:08:11.128828+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T17:08:11.147694+0000","last_undegraded":"2024-04-25T17:08:11.147694+0000","last_fullsized":"2024-04-25T17:08:11.147694+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.176383+0000","last_change":"2024-04-25T17:08:12.174095+0000","last_active":"2024-04-25T17:08:12.176383+0000","last_peered":"2024-04-25T17:08:12.176383+0000","last_clean":"2024-04-25T17:08:12.176383+0000","last_became_active":"2024-04-25T17:08:12.172786+0000","last_became_peered":"2024-04-25T17:08:12.172786+0000","last_unstale":"2024-04-25T17:08:12.176383+0000","last_undegraded":"2024-04-25T17:08:12.176383+0000","last_fullsized":"2024-04-25T17:08:12.176383+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:39.264530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495785+0000","last_change":"2024-04-25T17:08:12.158772+0000","last_active":"2024-04-25T17:08:13.495785+0000","last_peered":"2024-04-25T17:08:13.495785+0000","last_clean":"2024-04-25T17:08:13.495785+0000","last_became_active":"2024-04-25T17:08:12.158417+0000","last_became_peered":"2024-04-25T17:08:12.158417+0000","last_unstale":"2024-04-25T17:08:13.495785+0000","last_undegraded":"2024-04-25T17:08:13.495785+0000","last_fullsized":"2024-04-25T17:08:13.495785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:21.837887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.163235+0000","last_change":"2024-04-25T17:08:12.162384+0000","last_active":"2024-04-25T17:08:12.163235+0000","last_peered":"2024-04-25T17:08:12.163235+0000","last_clean":"2024-04-25T17:08:12.163235+0000","last_became_active":"2024-04-25T17:08:12.161997+0000","last_became_peered":"2024-04-25T17:08:12.161997+0000","last_unstale":"2024-04-25T17:08:12.163235+0000","last_undegraded":"2024-04-25T17:08:12.163235+0000","last_fullsized":"2024-04-25T17:08:12.163235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:19.759938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T17:08:12.160211+0000","last_change":"2024-04-25T17:08:12.159015+0000","last_active":"2024-04-25T17:08:12.160211+0000","last_peered":"2024-04-25T17:08:12.160211+0000","last_clean":"2024-04-25T17:08:12.160211+0000","last_became_active":"2024-04-25T17:08:12.157999+0000","last_became_peered":"2024-04-25T17:08:12.157999+0000","last_unstale":"2024-04-25T17:08:12.160211+0000","last_undegraded":"2024-04-25T17:08:12.160211+0000","last_fullsized":"2024-04-25T17:08:12.160211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:15.456131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280965,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":43629,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":371473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874258,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":981294,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037784,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":802816,"data_stored":381132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":819200,"data_stored":389724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":864256,"data_stored":980070,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":798720,"data_stored":380550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":811008,"data_stored":382162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":815104,"data_stored":389210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":410049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":371473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":802816,"data_stored":380572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":835584,"data_stored":961841,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38227,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T17:08:11.146203+0000', 'last_change': '2024-04-25T17:08:11.146203+0000', 'last_active': '2024-04-25T17:08:11.128828+0000', 'last_peered': '2024-04-25T17:08:11.128828+0000', 'last_clean': '2024-04-25T17:08:11.128828+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T17:08:11.146203+0000', 'last_undegraded': '2024-04-25T17:08:11.146203+0000', 'last_fullsized': '2024-04-25T17:08:11.146203+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_clean_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4, 1], 'acting': [5, 4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T17:08:11.144854+0000', 'last_change': '2024-04-25T17:08:11.144854+0000', 'last_active': '2024-04-25T17:08:11.128828+0000', 'last_peered': '2024-04-25T17:08:11.128828+0000', 'last_clean': '2024-04-25T17:08:11.128828+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T17:08:11.144854+0000', 'last_undegraded': '2024-04-25T17:08:11.144854+0000', 'last_fullsized': '2024-04-25T17:08:11.144854+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_clean_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4, 3], 'acting': [5, 4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T17:08:11.143338+0000', 'last_change': '2024-04-25T17:08:11.143338+0000', 'last_active': '2024-04-25T17:08:11.128828+0000', 'last_peered': '2024-04-25T17:08:11.128828+0000', 'last_clean': '2024-04-25T17:08:11.128828+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T17:08:11.143338+0000', 'last_undegraded': '2024-04-25T17:08:11.143338+0000', 'last_fullsized': '2024-04-25T17:08:11.143338+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_clean_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4, 1], 'acting': [5, 4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T17:08:11.149194+0000', 'last_change': '2024-04-25T17:08:11.149194+0000', 'last_active': '2024-04-25T17:08:11.128828+0000', 'last_peered': '2024-04-25T17:08:11.128828+0000', 'last_clean': '2024-04-25T17:08:11.128828+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T17:08:11.149194+0000', 'last_undegraded': '2024-04-25T17:08:11.149194+0000', 'last_fullsized': '2024-04-25T17:08:11.149194+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_clean_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2, 4], 'acting': [5, 2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-04-25T17:08:18.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 74, 'state': 'creating+peering', 'last_fresh': '2024-04-25T17:08:11.147694+0000', 'last_change': '2024-04-25T17:08:11.147694+0000', 'last_active': '2024-04-25T17:08:11.128828+0000', 'last_peered': '2024-04-25T17:08:11.128828+0000', 'last_clean': '2024-04-25T17:08:11.128828+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T17:08:11.147694+0000', 'last_undegraded': '2024-04-25T17:08:11.147694+0000', 'last_fullsized': '2024-04-25T17:08:11.147694+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'last_clean_scrub_stamp': '2024-04-25T17:08:11.128828+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 9, 3], 'acting': [5, 9, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T17:08:18.924 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T17:08:19.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 -- 172.21.15.96:0/2721712664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43241080c0 msgr2=0x7f432410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 --2- 172.21.15.96:0/2721712664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43241080c0 0x7f432410a4e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f431c009ca0 tx=0x7f431c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 -- 172.21.15.96:0/2721712664 shutdown_connections 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 --2- 172.21.15.96:0/2721712664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43241080c0 0x7f432410a4e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 --2- 172.21.15.96:0/2721712664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43241033c0 0x7f43241037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 --2- 172.21.15.96:0/2721712664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4324102180 0x7f4324102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 -- 172.21.15.96:0/2721712664 >> 172.21.15.96:0/2721712664 conn(0x7f43240fd370 msgr2=0x7f43240ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 -- 172.21.15.96:0/2721712664 shutdown_connections 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.064+0000 7f4329a84640 1 -- 172.21.15.96:0/2721712664 wait complete. 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 Processor -- start 2024-04-25T17:08:19.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 -- start start 2024-04-25T17:08:19.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4324102180 0x7f4324109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43241033c0 0x7f432419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43241080c0 0x7f43241a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f432410dca0 con 0x7f4324102180 2024-04-25T17:08:19.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f432410db20 con 0x7f43241080c0 2024-04-25T17:08:19.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4329a84640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f432410de20 con 0x7f43241033c0 2024-04-25T17:08:19.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f43237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4324102180 0x7f4324109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f43237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4324102180 0x7f4324109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47672/0 (socket says 172.21.15.96:47672) 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f43237fe640 1 -- 172.21.15.96:0/1869148905 learned_addr learned my addr 172.21.15.96:0/1869148905 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4322ffd640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43241033c0 0x7f432419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.065+0000 7f4323fff640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43241080c0 0x7f43241a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f43237fe640 1 -- 172.21.15.96:0/1869148905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43241033c0 msgr2=0x7f432419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f43237fe640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43241033c0 0x7f432419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f43237fe640 1 -- 172.21.15.96:0/1869148905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43241080c0 msgr2=0x7f43241a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f43237fe640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43241080c0 0x7f43241a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f43237fe640 1 -- 172.21.15.96:0/1869148905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43241a1d40 con 0x7f4324102180 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f43237fe640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4324102180 0x7f4324109fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4314005d00 tx=0x7f431400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f4320ff9640 1 -- 172.21.15.96:0/1869148905 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314017070 con 0x7f4324102180 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f4320ff9640 1 -- 172.21.15.96:0/1869148905 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314009c30 con 0x7f4324102180 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43241a1fd0 con 0x7f4324102180 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f4320ff9640 1 -- 172.21.15.96:0/1869148905 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431400f620 con 0x7f4324102180 2024-04-25T17:08:19.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.066+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43241a2440 con 0x7f4324102180 2024-04-25T17:08:19.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.067+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43241077c0 con 0x7f4324102180 2024-04-25T17:08:19.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.068+0000 7f4320ff9640 1 -- 172.21.15.96:0/1869148905 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f431401a070 con 0x7f4324102180 2024-04-25T17:08:19.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.069+0000 7f4320ff9640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f430c0afb80 0x7f430c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.069+0000 7f4322ffd640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f430c0afb80 0x7f430c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.069+0000 7f4320ff9640 1 -- 172.21.15.96:0/1869148905 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f43140de5b0 con 0x7f4324102180 2024-04-25T17:08:19.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.070+0000 7f4322ffd640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f430c0afb80 0x7f430c0b2090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4324109950 tx=0x7f431800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:19.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.073+0000 7f4320ff9640 1 -- 172.21.15.96:0/1869148905 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4314009de0 con 0x7f4324102180 2024-04-25T17:08:19.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.321+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f432402db50 con 0x7f4324102180 2024-04-25T17:08:19.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.322+0000 7f4320ff9640 1 -- 172.21.15.96:0/1869148905 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2499 (secure 0 0 0) 0x7f43140af260 con 0x7f4324102180 2024-04-25T17:08:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:19.322 INFO:teuthology.orchestra.run.smithi096.stdout:{"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":378,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1714064841,"num_in_osds":12,"osd_in_since":1714064820,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592618,"bytes_used":367443968,"bytes_avail":1151572983808,"bytes_total":1151940427776,"inactive_pgs_ratio":0.051546391099691391,"read_bytes_sec":682,"write_bytes_sec":2047,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":18,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24485},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24521}],"up:standby":10},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.96:8443/"}},"servicemap":{"epoch":17,"modified":"2024-04-25T17:07:55.045041+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"j":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","11":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f430c0afb80 msgr2=0x7f430c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f430c0afb80 0x7f430c0b2090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4324109950 tx=0x7f431800b390 comp rx=0 tx=0).stop 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4324102180 msgr2=0x7f4324109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4324102180 0x7f4324109fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4314005d00 tx=0x7f431400dde0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 shutdown_connections 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43241080c0 0x7f43241a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f430c0afb80 0x7f430c0b2090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43241033c0 0x7f432419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 --2- 172.21.15.96:0/1869148905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4324102180 0x7f4324109fb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 >> 172.21.15.96:0/1869148905 conn(0x7f43240fd370 msgr2=0x7f43240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 shutdown_connections 2024-04-25T17:08:19.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.324+0000 7f4329a84640 1 -- 172.21.15.96:0/1869148905 wait complete. 2024-04-25T17:08:19.335 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T17:08:19.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.443+0000 7f6fa0397640 1 -- 172.21.15.96:0/4023142679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f98101ef0 msgr2=0x7f6f98106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.443+0000 7f6fa0397640 1 --2- 172.21.15.96:0/4023142679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f98101ef0 0x7f6f98106b60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009ca0 tx=0x7f6f9402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- 172.21.15.96:0/4023142679 shutdown_connections 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 --2- 172.21.15.96:0/4023142679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f98109a00 0x7f6f9810be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 --2- 172.21.15.96:0/4023142679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f981070a0 0x7f6f981094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 --2- 172.21.15.96:0/4023142679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f98101ef0 0x7f6f98106b60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- 172.21.15.96:0/4023142679 >> 172.21.15.96:0/4023142679 conn(0x7f6f980fd330 msgr2=0x7f6f980ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- 172.21.15.96:0/4023142679 shutdown_connections 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- 172.21.15.96:0/4023142679 wait complete. 2024-04-25T17:08:19.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 Processor -- start 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- start start 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f98101ef0 0x7f6f9819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f981070a0 0x7f6f9819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f98109a00 0x7f6f981a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810f670 con 0x7f6f981070a0 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810f4f0 con 0x7f6f98101ef0 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.444+0000 7f6fa0397640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810f7f0 con 0x7f6f98109a00 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f981070a0 0x7f6f9819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f981070a0 0x7f6f9819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47690/0 (socket says 172.21.15.96:47690) 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 -- 172.21.15.96:0/1960968013 learned_addr learned my addr 172.21.15.96:0/1960968013 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9e133640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f98101ef0 0x7f6f9819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9e934640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f98109a00 0x7f6f981a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 -- 172.21.15.96:0/1960968013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f98109a00 msgr2=0x7f6f981a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f98109a00 0x7f6f981a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 -- 172.21.15.96:0/1960968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f98101ef0 msgr2=0x7f6f9819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f98101ef0 0x7f6f9819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 -- 172.21.15.96:0/1960968013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f981a1d70 con 0x7f6f981070a0 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9e934640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f98109a00 0x7f6f981a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f9d932640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f981070a0 0x7f6f9819a820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c970 tx=0x7f6f8800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f8f7fe640 1 -- 172.21.15.96:0/1960968013 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88007bb0 con 0x7f6f981070a0 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f8f7fe640 1 -- 172.21.15.96:0/1960968013 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f88007d50 con 0x7f6f981070a0 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f981a2060 con 0x7f6f981070a0 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6f8f7fe640 1 -- 172.21.15.96:0/1960968013 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800f450 con 0x7f6f981070a0 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.445+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f981a25b0 con 0x7f6f981070a0 2024-04-25T17:08:19.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.446+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f980fee50 con 0x7f6f981070a0 2024-04-25T17:08:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.448+0000 7f6f8f7fe640 1 -- 172.21.15.96:0/1960968013 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6f88004ee0 con 0x7f6f981070a0 2024-04-25T17:08:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.449+0000 7f6f8f7fe640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6f840afc00 0x7f6f840b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.449+0000 7f6f9e133640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6f840afc00 0x7f6f840b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.449+0000 7f6f8f7fe640 1 -- 172.21.15.96:0/1960968013 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f6f880dea00 con 0x7f6f981070a0 2024-04-25T17:08:19.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.449+0000 7f6f9e133640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6f840afc00 0x7f6f840b2110 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000f80 tx=0x7f6f94009420 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:19.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.452+0000 7f6f8f7fe640 1 -- 172.21.15.96:0/1960968013 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f88014030 con 0x7f6f981070a0 2024-04-25T17:08:19.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.628+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f98062fe0 con 0x7f6f840afc00 2024-04-25T17:08:19.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.635+0000 7f6f8f7fe640 1 -- 172.21.15.96:0/1960968013 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258867 (secure 0 0 0) 0x7f6f98062fe0 con 0x7f6f840afc00 2024-04-25T17:08:19.638 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:19.639 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T17:08:19.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6f840afc00 msgr2=0x7f6f840b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6f840afc00 0x7f6f840b2110 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f94000f80 tx=0x7f6f94009420 comp rx=0 tx=0).stop 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f981070a0 msgr2=0x7f6f9819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f981070a0 0x7f6f9819a820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c970 tx=0x7f6f8800ce30 comp rx=0 tx=0).stop 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 shutdown_connections 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6f840afc00 0x7f6f840b2110 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f98109a00 0x7f6f981a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f981070a0 0x7f6f9819a820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 --2- 172.21.15.96:0/1960968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f98101ef0 0x7f6f9819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.638+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 >> 172.21.15.96:0/1960968013 conn(0x7f6f980fd330 msgr2=0x7f6f98105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.639+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 shutdown_connections 2024-04-25T17:08:19.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:19.639+0000 7f6fa0397640 1 -- 172.21.15.96:0/1960968013 wait complete. 2024-04-25T17:08:19.648 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:19 smithi096 ceph-mon[29131]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:19 smithi096 ceph-mon[29131]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:19 smithi096 ceph-mon[29131]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:19 smithi096 ceph-mon[29131]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 776 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T17:08:19.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:19 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1869148905' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-25T17:08:19.650 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-04-25T17:08:19.050677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":87,"num_read_kb":75,"num_write":172,"num_write_kb":2166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":273,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358992,"kb_used_data":9800,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582832,"statfs":{"total":1151940427776,"available":1151572819968,"internally_reserved":0,"allocated":10035200,"data_stored":6503872,"data_compressed":109033,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":49,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.915118"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141585+0000","last_change":"2024-04-25T17:08:12.162430+0000","last_active":"2024-04-25T17:08:13.141585+0000","last_peered":"2024-04-25T17:08:13.141585+0000","last_clean":"2024-04-25T17:08:13.141585+0000","last_became_active":"2024-04-25T17:08:12.162052+0000","last_became_peered":"2024-04-25T17:08:12.162052+0000","last_unstale":"2024-04-25T17:08:13.141585+0000","last_undegraded":"2024-04-25T17:08:13.141585+0000","last_fullsized":"2024-04-25T17:08:13.141585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:09.637937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421957+0000","last_change":"2024-04-25T17:08:12.169139+0000","last_active":"2024-04-25T17:08:13.421957+0000","last_peered":"2024-04-25T17:08:13.421957+0000","last_clean":"2024-04-25T17:08:13.421957+0000","last_became_active":"2024-04-25T17:08:12.168715+0000","last_became_peered":"2024-04-25T17:08:12.168715+0000","last_unstale":"2024-04-25T17:08:13.421957+0000","last_undegraded":"2024-04-25T17:08:13.421957+0000","last_fullsized":"2024-04-25T17:08:13.421957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:57.357750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421733+0000","last_change":"2024-04-25T17:08:12.159141+0000","last_active":"2024-04-25T17:08:13.421733+0000","last_peered":"2024-04-25T17:08:13.421733+0000","last_clean":"2024-04-25T17:08:13.421733+0000","last_became_active":"2024-04-25T17:08:12.158796+0000","last_became_peered":"2024-04-25T17:08:12.158796+0000","last_unstale":"2024-04-25T17:08:13.421733+0000","last_undegraded":"2024-04-25T17:08:13.421733+0000","last_fullsized":"2024-04-25T17:08:13.421733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:55.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495311+0000","last_change":"2024-04-25T17:08:12.160464+0000","last_active":"2024-04-25T17:08:13.495311+0000","last_peered":"2024-04-25T17:08:13.495311+0000","last_clean":"2024-04-25T17:08:13.495311+0000","last_became_active":"2024-04-25T17:08:12.160118+0000","last_became_peered":"2024-04-25T17:08:12.160118+0000","last_unstale":"2024-04-25T17:08:13.495311+0000","last_undegraded":"2024-04-25T17:08:13.495311+0000","last_fullsized":"2024-04-25T17:08:13.495311+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:59:06.465964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495515+0000","last_change":"2024-04-25T17:08:12.174591+0000","last_active":"2024-04-25T17:08:13.495515+0000","last_peered":"2024-04-25T17:08:13.495515+0000","last_clean":"2024-04-25T17:08:13.495515+0000","last_became_active":"2024-04-25T17:08:12.174173+0000","last_became_peered":"2024-04-25T17:08:12.174173+0000","last_unstale":"2024-04-25T17:08:13.495515+0000","last_undegraded":"2024-04-25T17:08:13.495515+0000","last_fullsized":"2024-04-25T17:08:13.495515+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:12.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141095+0000","last_change":"2024-04-25T17:08:12.164230+0000","last_active":"2024-04-25T17:08:13.141095+0000","last_peered":"2024-04-25T17:08:13.141095+0000","last_clean":"2024-04-25T17:08:13.141095+0000","last_became_active":"2024-04-25T17:08:12.163868+0000","last_became_peered":"2024-04-25T17:08:12.163868+0000","last_unstale":"2024-04-25T17:08:13.141095+0000","last_undegraded":"2024-04-25T17:08:13.141095+0000","last_fullsized":"2024-04-25T17:08:13.141095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:12.604164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121611+0000","last_change":"2024-04-25T17:08:12.161627+0000","last_active":"2024-04-25T17:08:14.121611+0000","last_peered":"2024-04-25T17:08:14.121611+0000","last_clean":"2024-04-25T17:08:14.121611+0000","last_became_active":"2024-04-25T17:08:12.161146+0000","last_became_peered":"2024-04-25T17:08:12.161146+0000","last_unstale":"2024-04-25T17:08:14.121611+0000","last_undegraded":"2024-04-25T17:08:14.121611+0000","last_fullsized":"2024-04-25T17:08:14.121611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.972053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140558+0000","last_change":"2024-04-25T17:08:12.165512+0000","last_active":"2024-04-25T17:08:13.140558+0000","last_peered":"2024-04-25T17:08:13.140558+0000","last_clean":"2024-04-25T17:08:13.140558+0000","last_became_active":"2024-04-25T17:08:12.164018+0000","last_became_peered":"2024-04-25T17:08:12.164018+0000","last_unstale":"2024-04-25T17:08:13.140558+0000","last_undegraded":"2024-04-25T17:08:13.140558+0000","last_fullsized":"2024-04-25T17:08:13.140558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:28.075786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495771+0000","last_change":"2024-04-25T17:08:12.173313+0000","last_active":"2024-04-25T17:08:13.495771+0000","last_peered":"2024-04-25T17:08:13.495771+0000","last_clean":"2024-04-25T17:08:13.495771+0000","last_became_active":"2024-04-25T17:08:12.171269+0000","last_became_peered":"2024-04-25T17:08:12.171269+0000","last_unstale":"2024-04-25T17:08:13.495771+0000","last_undegraded":"2024-04-25T17:08:13.495771+0000","last_fullsized":"2024-04-25T17:08:13.495771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:39.565973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141559+0000","last_change":"2024-04-25T17:08:12.166114+0000","last_active":"2024-04-25T17:08:13.141559+0000","last_peered":"2024-04-25T17:08:13.141559+0000","last_clean":"2024-04-25T17:08:13.141559+0000","last_became_active":"2024-04-25T17:08:12.165737+0000","last_became_peered":"2024-04-25T17:08:12.165737+0000","last_unstale":"2024-04-25T17:08:13.141559+0000","last_undegraded":"2024-04-25T17:08:13.141559+0000","last_fullsized":"2024-04-25T17:08:13.141559+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:55.624239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122116+0000","last_change":"2024-04-25T17:08:12.160435+0000","last_active":"2024-04-25T17:08:14.122116+0000","last_peered":"2024-04-25T17:08:14.122116+0000","last_clean":"2024-04-25T17:08:14.122116+0000","last_became_active":"2024-04-25T17:08:12.160085+0000","last_became_peered":"2024-04-25T17:08:12.160085+0000","last_unstale":"2024-04-25T17:08:14.122116+0000","last_undegraded":"2024-04-25T17:08:14.122116+0000","last_fullsized":"2024-04-25T17:08:14.122116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.709567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495078+0000","last_change":"2024-04-25T17:08:12.173990+0000","last_active":"2024-04-25T17:08:13.495078+0000","last_peered":"2024-04-25T17:08:13.495078+0000","last_clean":"2024-04-25T17:08:13.495078+0000","last_became_active":"2024-04-25T17:08:12.172838+0000","last_became_peered":"2024-04-25T17:08:12.172838+0000","last_unstale":"2024-04-25T17:08:13.495078+0000","last_undegraded":"2024-04-25T17:08:13.495078+0000","last_fullsized":"2024-04-25T17:08:13.495078+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:00:56.720758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495615+0000","last_change":"2024-04-25T17:08:12.167579+0000","last_active":"2024-04-25T17:08:13.495615+0000","last_peered":"2024-04-25T17:08:13.495615+0000","last_clean":"2024-04-25T17:08:13.495615+0000","last_became_active":"2024-04-25T17:08:12.166583+0000","last_became_peered":"2024-04-25T17:08:12.166583+0000","last_unstale":"2024-04-25T17:08:13.495615+0000","last_undegraded":"2024-04-25T17:08:13.495615+0000","last_fullsized":"2024-04-25T17:08:13.495615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:44.081814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421592+0000","last_change":"2024-04-25T17:08:12.161263+0000","last_active":"2024-04-25T17:08:13.421592+0000","last_peered":"2024-04-25T17:08:13.421592+0000","last_clean":"2024-04-25T17:08:13.421592+0000","last_became_active":"2024-04-25T17:08:12.160078+0000","last_became_peered":"2024-04-25T17:08:12.160078+0000","last_unstale":"2024-04-25T17:08:13.421592+0000","last_undegraded":"2024-04-25T17:08:13.421592+0000","last_fullsized":"2024-04-25T17:08:13.421592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:56.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421594+0000","last_change":"2024-04-25T17:08:12.167318+0000","last_active":"2024-04-25T17:08:13.421594+0000","last_peered":"2024-04-25T17:08:13.421594+0000","last_clean":"2024-04-25T17:08:13.421594+0000","last_became_active":"2024-04-25T17:08:12.166964+0000","last_became_peered":"2024-04-25T17:08:12.166964+0000","last_unstale":"2024-04-25T17:08:13.421594+0000","last_undegraded":"2024-04-25T17:08:13.421594+0000","last_fullsized":"2024-04-25T17:08:13.421594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:25.967818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494787+0000","last_change":"2024-04-25T17:08:12.166417+0000","last_active":"2024-04-25T17:08:13.494787+0000","last_peered":"2024-04-25T17:08:13.494787+0000","last_clean":"2024-04-25T17:08:13.494787+0000","last_became_active":"2024-04-25T17:08:12.165928+0000","last_became_peered":"2024-04-25T17:08:12.165928+0000","last_unstale":"2024-04-25T17:08:13.494787+0000","last_undegraded":"2024-04-25T17:08:13.494787+0000","last_fullsized":"2024-04-25T17:08:13.494787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:24:22.359814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141635+0000","last_change":"2024-04-25T17:08:12.164952+0000","last_active":"2024-04-25T17:08:13.141635+0000","last_peered":"2024-04-25T17:08:13.141635+0000","last_clean":"2024-04-25T17:08:13.141635+0000","last_became_active":"2024-04-25T17:08:12.164595+0000","last_became_peered":"2024-04-25T17:08:12.164595+0000","last_unstale":"2024-04-25T17:08:13.141635+0000","last_undegraded":"2024-04-25T17:08:13.141635+0000","last_fullsized":"2024-04-25T17:08:13.141635+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:45.020512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495412+0000","last_change":"2024-04-25T17:08:12.153583+0000","last_active":"2024-04-25T17:08:13.495412+0000","last_peered":"2024-04-25T17:08:13.495412+0000","last_clean":"2024-04-25T17:08:13.495412+0000","last_became_active":"2024-04-25T17:08:12.153070+0000","last_became_peered":"2024-04-25T17:08:12.153070+0000","last_unstale":"2024-04-25T17:08:13.495412+0000","last_undegraded":"2024-04-25T17:08:13.495412+0000","last_fullsized":"2024-04-25T17:08:13.495412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:34.151002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352707+0000","last_change":"2024-04-25T17:08:12.166837+0000","last_active":"2024-04-25T17:08:13.352707+0000","last_peered":"2024-04-25T17:08:13.352707+0000","last_clean":"2024-04-25T17:08:13.352707+0000","last_became_active":"2024-04-25T17:08:12.166477+0000","last_became_peered":"2024-04-25T17:08:12.166477+0000","last_unstale":"2024-04-25T17:08:13.352707+0000","last_undegraded":"2024-04-25T17:08:13.352707+0000","last_fullsized":"2024-04-25T17:08:13.352707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:50.492333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495157+0000","last_change":"2024-04-25T17:08:11.153164+0000","last_active":"2024-04-25T17:08:13.495157+0000","last_peered":"2024-04-25T17:08:13.495157+0000","last_clean":"2024-04-25T17:08:13.495157+0000","last_became_active":"2024-04-25T17:08:11.152189+0000","last_became_peered":"2024-04-25T17:08:11.152189+0000","last_unstale":"2024-04-25T17:08:13.495157+0000","last_undegraded":"2024-04-25T17:08:13.495157+0000","last_fullsized":"2024-04-25T17:08:13.495157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:30.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141002+0000","last_change":"2024-04-25T17:08:12.178117+0000","last_active":"2024-04-25T17:08:13.141002+0000","last_peered":"2024-04-25T17:08:13.141002+0000","last_clean":"2024-04-25T17:08:13.141002+0000","last_became_active":"2024-04-25T17:08:12.177767+0000","last_became_peered":"2024-04-25T17:08:12.177767+0000","last_unstale":"2024-04-25T17:08:13.141002+0000","last_undegraded":"2024-04-25T17:08:13.141002+0000","last_fullsized":"2024-04-25T17:08:13.141002+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:34.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141096+0000","last_change":"2024-04-25T17:08:11.153996+0000","last_active":"2024-04-25T17:08:13.141096+0000","last_peered":"2024-04-25T17:08:13.141096+0000","last_clean":"2024-04-25T17:08:13.141096+0000","last_became_active":"2024-04-25T17:08:11.152993+0000","last_became_peered":"2024-04-25T17:08:11.152993+0000","last_unstale":"2024-04-25T17:08:13.141096+0000","last_undegraded":"2024-04-25T17:08:13.141096+0000","last_fullsized":"2024-04-25T17:08:13.141096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:33.116616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422141+0000","last_change":"2024-04-25T17:08:12.163707+0000","last_active":"2024-04-25T17:08:13.422141+0000","last_peered":"2024-04-25T17:08:13.422141+0000","last_clean":"2024-04-25T17:08:13.422141+0000","last_became_active":"2024-04-25T17:08:12.161868+0000","last_became_peered":"2024-04-25T17:08:12.161868+0000","last_unstale":"2024-04-25T17:08:13.422141+0000","last_undegraded":"2024-04-25T17:08:13.422141+0000","last_fullsized":"2024-04-25T17:08:13.422141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.333036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496262+0000","last_change":"2024-04-25T17:08:11.158101+0000","last_active":"2024-04-25T17:08:13.496262+0000","last_peered":"2024-04-25T17:08:13.496262+0000","last_clean":"2024-04-25T17:08:13.496262+0000","last_became_active":"2024-04-25T17:08:11.157747+0000","last_became_peered":"2024-04-25T17:08:11.157747+0000","last_unstale":"2024-04-25T17:08:13.496262+0000","last_undegraded":"2024-04-25T17:08:13.496262+0000","last_fullsized":"2024-04-25T17:08:13.496262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:03.840823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495644+0000","last_change":"2024-04-25T17:08:12.170278+0000","last_active":"2024-04-25T17:08:13.495644+0000","last_peered":"2024-04-25T17:08:13.495644+0000","last_clean":"2024-04-25T17:08:13.495644+0000","last_became_active":"2024-04-25T17:08:12.169219+0000","last_became_peered":"2024-04-25T17:08:12.169219+0000","last_unstale":"2024-04-25T17:08:13.495644+0000","last_undegraded":"2024-04-25T17:08:13.495644+0000","last_fullsized":"2024-04-25T17:08:13.495644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:35.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496183+0000","last_change":"2024-04-25T17:08:11.153167+0000","last_active":"2024-04-25T17:08:13.496183+0000","last_peered":"2024-04-25T17:08:13.496183+0000","last_clean":"2024-04-25T17:08:13.496183+0000","last_became_active":"2024-04-25T17:08:11.152772+0000","last_became_peered":"2024-04-25T17:08:11.152772+0000","last_unstale":"2024-04-25T17:08:13.496183+0000","last_undegraded":"2024-04-25T17:08:13.496183+0000","last_fullsized":"2024-04-25T17:08:13.496183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:57.198963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495439+0000","last_change":"2024-04-25T17:08:12.163530+0000","last_active":"2024-04-25T17:08:13.495439+0000","last_peered":"2024-04-25T17:08:13.495439+0000","last_clean":"2024-04-25T17:08:13.495439+0000","last_became_active":"2024-04-25T17:08:12.163179+0000","last_became_peered":"2024-04-25T17:08:12.163179+0000","last_unstale":"2024-04-25T17:08:13.495439+0000","last_undegraded":"2024-04-25T17:08:13.495439+0000","last_fullsized":"2024-04-25T17:08:13.495439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:59.536274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495764+0000","last_change":"2024-04-25T17:08:11.158564+0000","last_active":"2024-04-25T17:08:13.495764+0000","last_peered":"2024-04-25T17:08:13.495764+0000","last_clean":"2024-04-25T17:08:13.495764+0000","last_became_active":"2024-04-25T17:08:11.158143+0000","last_became_peered":"2024-04-25T17:08:11.158143+0000","last_unstale":"2024-04-25T17:08:13.495764+0000","last_undegraded":"2024-04-25T17:08:13.495764+0000","last_fullsized":"2024-04-25T17:08:13.495764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:03.957700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495878+0000","last_change":"2024-04-25T17:08:12.167584+0000","last_active":"2024-04-25T17:08:13.495878+0000","last_peered":"2024-04-25T17:08:13.495878+0000","last_clean":"2024-04-25T17:08:13.495878+0000","last_became_active":"2024-04-25T17:08:12.166717+0000","last_became_peered":"2024-04-25T17:08:12.166717+0000","last_unstale":"2024-04-25T17:08:13.495878+0000","last_undegraded":"2024-04-25T17:08:13.495878+0000","last_fullsized":"2024-04-25T17:08:13.495878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:38.047671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141009+0000","last_change":"2024-04-25T17:08:11.159254+0000","last_active":"2024-04-25T17:08:13.141009+0000","last_peered":"2024-04-25T17:08:13.141009+0000","last_clean":"2024-04-25T17:08:13.141009+0000","last_became_active":"2024-04-25T17:08:11.158864+0000","last_became_peered":"2024-04-25T17:08:11.158864+0000","last_unstale":"2024-04-25T17:08:13.141009+0000","last_undegraded":"2024-04-25T17:08:13.141009+0000","last_fullsized":"2024-04-25T17:08:13.141009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:17.795010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495942+0000","last_change":"2024-04-25T17:08:12.170409+0000","last_active":"2024-04-25T17:08:13.495942+0000","last_peered":"2024-04-25T17:08:13.495942+0000","last_clean":"2024-04-25T17:08:13.495942+0000","last_became_active":"2024-04-25T17:08:12.169510+0000","last_became_peered":"2024-04-25T17:08:12.169510+0000","last_unstale":"2024-04-25T17:08:13.495942+0000","last_undegraded":"2024-04-25T17:08:13.495942+0000","last_fullsized":"2024-04-25T17:08:13.495942+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.310486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495182+0000","last_change":"2024-04-25T17:08:11.150980+0000","last_active":"2024-04-25T17:08:13.495182+0000","last_peered":"2024-04-25T17:08:13.495182+0000","last_clean":"2024-04-25T17:08:13.495182+0000","last_became_active":"2024-04-25T17:08:11.150580+0000","last_became_peered":"2024-04-25T17:08:11.150580+0000","last_unstale":"2024-04-25T17:08:13.495182+0000","last_undegraded":"2024-04-25T17:08:13.495182+0000","last_fullsized":"2024-04-25T17:08:13.495182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:59.539673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140605+0000","last_change":"2024-04-25T17:08:12.171013+0000","last_active":"2024-04-25T17:08:13.140605+0000","last_peered":"2024-04-25T17:08:13.140605+0000","last_clean":"2024-04-25T17:08:13.140605+0000","last_became_active":"2024-04-25T17:08:12.169868+0000","last_became_peered":"2024-04-25T17:08:12.169868+0000","last_unstale":"2024-04-25T17:08:13.140605+0000","last_undegraded":"2024-04-25T17:08:13.140605+0000","last_fullsized":"2024-04-25T17:08:13.140605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:22.418990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495287+0000","last_change":"2024-04-25T17:08:11.157618+0000","last_active":"2024-04-25T17:08:13.495287+0000","last_peered":"2024-04-25T17:08:13.495287+0000","last_clean":"2024-04-25T17:08:13.495287+0000","last_became_active":"2024-04-25T17:08:11.157052+0000","last_became_peered":"2024-04-25T17:08:11.157052+0000","last_unstale":"2024-04-25T17:08:13.495287+0000","last_undegraded":"2024-04-25T17:08:13.495287+0000","last_fullsized":"2024-04-25T17:08:13.495287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:06.032163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494806+0000","last_change":"2024-04-25T17:08:12.151093+0000","last_active":"2024-04-25T17:08:13.494806+0000","last_peered":"2024-04-25T17:08:13.494806+0000","last_clean":"2024-04-25T17:08:13.494806+0000","last_became_active":"2024-04-25T17:08:12.150699+0000","last_became_peered":"2024-04-25T17:08:12.150699+0000","last_unstale":"2024-04-25T17:08:13.494806+0000","last_undegraded":"2024-04-25T17:08:13.494806+0000","last_fullsized":"2024-04-25T17:08:13.494806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:27.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141389+0000","last_change":"2024-04-25T17:08:11.157673+0000","last_active":"2024-04-25T17:08:13.141389+0000","last_peered":"2024-04-25T17:08:13.141389+0000","last_clean":"2024-04-25T17:08:13.141389+0000","last_became_active":"2024-04-25T17:08:11.157089+0000","last_became_peered":"2024-04-25T17:08:11.157089+0000","last_unstale":"2024-04-25T17:08:13.141389+0000","last_undegraded":"2024-04-25T17:08:13.141389+0000","last_fullsized":"2024-04-25T17:08:13.141389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:47.586003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495779+0000","last_change":"2024-04-25T17:08:12.170666+0000","last_active":"2024-04-25T17:08:13.495779+0000","last_peered":"2024-04-25T17:08:13.495779+0000","last_clean":"2024-04-25T17:08:13.495779+0000","last_became_active":"2024-04-25T17:08:12.169607+0000","last_became_peered":"2024-04-25T17:08:12.169607+0000","last_unstale":"2024-04-25T17:08:13.495779+0000","last_undegraded":"2024-04-25T17:08:13.495779+0000","last_fullsized":"2024-04-25T17:08:13.495779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121887+0000","last_change":"2024-04-25T17:08:11.153407+0000","last_active":"2024-04-25T17:08:14.121887+0000","last_peered":"2024-04-25T17:08:14.121887+0000","last_clean":"2024-04-25T17:08:14.121887+0000","last_became_active":"2024-04-25T17:08:11.152932+0000","last_became_peered":"2024-04-25T17:08:11.152932+0000","last_unstale":"2024-04-25T17:08:14.121887+0000","last_undegraded":"2024-04-25T17:08:14.121887+0000","last_fullsized":"2024-04-25T17:08:14.121887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:06.712620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422086+0000","last_change":"2024-04-25T17:08:12.173788+0000","last_active":"2024-04-25T17:08:13.422086+0000","last_peered":"2024-04-25T17:08:13.422086+0000","last_clean":"2024-04-25T17:08:13.422086+0000","last_became_active":"2024-04-25T17:08:12.172714+0000","last_became_peered":"2024-04-25T17:08:12.172714+0000","last_unstale":"2024-04-25T17:08:13.422086+0000","last_undegraded":"2024-04-25T17:08:13.422086+0000","last_fullsized":"2024-04-25T17:08:13.422086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:18.892187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495573+0000","last_change":"2024-04-25T17:08:11.159802+0000","last_active":"2024-04-25T17:08:13.495573+0000","last_peered":"2024-04-25T17:08:13.495573+0000","last_clean":"2024-04-25T17:08:13.495573+0000","last_became_active":"2024-04-25T17:08:11.159431+0000","last_became_peered":"2024-04-25T17:08:11.159431+0000","last_unstale":"2024-04-25T17:08:13.495573+0000","last_undegraded":"2024-04-25T17:08:13.495573+0000","last_fullsized":"2024-04-25T17:08:13.495573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495822+0000","last_change":"2024-04-25T17:08:12.162703+0000","last_active":"2024-04-25T17:08:13.495822+0000","last_peered":"2024-04-25T17:08:13.495822+0000","last_clean":"2024-04-25T17:08:13.495822+0000","last_became_active":"2024-04-25T17:08:12.162361+0000","last_became_peered":"2024-04-25T17:08:12.162361+0000","last_unstale":"2024-04-25T17:08:13.495822+0000","last_undegraded":"2024-04-25T17:08:13.495822+0000","last_fullsized":"2024-04-25T17:08:13.495822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:51.262319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'6","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141260+0000","last_change":"2024-04-25T17:08:11.153938+0000","last_active":"2024-04-25T17:08:13.141260+0000","last_peered":"2024-04-25T17:08:13.141260+0000","last_clean":"2024-04-25T17:08:13.141260+0000","last_became_active":"2024-04-25T17:08:11.153591+0000","last_became_peered":"2024-04-25T17:08:11.153591+0000","last_unstale":"2024-04-25T17:08:13.141260+0000","last_undegraded":"2024-04-25T17:08:13.141260+0000","last_fullsized":"2024-04-25T17:08:13.141260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496204+0000","last_change":"2024-04-25T17:08:12.156045+0000","last_active":"2024-04-25T17:08:13.496204+0000","last_peered":"2024-04-25T17:08:13.496204+0000","last_clean":"2024-04-25T17:08:13.496204+0000","last_became_active":"2024-04-25T17:08:12.155657+0000","last_became_peered":"2024-04-25T17:08:12.155657+0000","last_unstale":"2024-04-25T17:08:13.496204+0000","last_undegraded":"2024-04-25T17:08:13.496204+0000","last_fullsized":"2024-04-25T17:08:13.496204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352496+0000","last_change":"2024-04-25T17:08:11.162009+0000","last_active":"2024-04-25T17:08:13.352496+0000","last_peered":"2024-04-25T17:08:13.352496+0000","last_clean":"2024-04-25T17:08:13.352496+0000","last_became_active":"2024-04-25T17:08:11.161574+0000","last_became_peered":"2024-04-25T17:08:11.161574+0000","last_unstale":"2024-04-25T17:08:13.352496+0000","last_undegraded":"2024-04-25T17:08:13.352496+0000","last_fullsized":"2024-04-25T17:08:13.352496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:16.760355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495191+0000","last_change":"2024-04-25T17:08:12.159522+0000","last_active":"2024-04-25T17:08:13.495191+0000","last_peered":"2024-04-25T17:08:13.495191+0000","last_clean":"2024-04-25T17:08:13.495191+0000","last_became_active":"2024-04-25T17:08:12.159036+0000","last_became_peered":"2024-04-25T17:08:12.159036+0000","last_unstale":"2024-04-25T17:08:13.495191+0000","last_undegraded":"2024-04-25T17:08:13.495191+0000","last_fullsized":"2024-04-25T17:08:13.495191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:32.044488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495796+0000","last_change":"2024-04-25T17:08:11.162487+0000","last_active":"2024-04-25T17:08:13.495796+0000","last_peered":"2024-04-25T17:08:13.495796+0000","last_clean":"2024-04-25T17:08:13.495796+0000","last_became_active":"2024-04-25T17:08:11.161961+0000","last_became_peered":"2024-04-25T17:08:11.161961+0000","last_unstale":"2024-04-25T17:08:13.495796+0000","last_undegraded":"2024-04-25T17:08:13.495796+0000","last_fullsized":"2024-04-25T17:08:13.495796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:02.643683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121943+0000","last_change":"2024-04-25T17:08:12.164107+0000","last_active":"2024-04-25T17:08:14.121943+0000","last_peered":"2024-04-25T17:08:14.121943+0000","last_clean":"2024-04-25T17:08:14.121943+0000","last_became_active":"2024-04-25T17:08:12.163741+0000","last_became_peered":"2024-04-25T17:08:12.163741+0000","last_unstale":"2024-04-25T17:08:14.121943+0000","last_undegraded":"2024-04-25T17:08:14.121943+0000","last_fullsized":"2024-04-25T17:08:14.121943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.552833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494622+0000","last_change":"2024-04-25T17:08:11.153383+0000","last_active":"2024-04-25T17:08:13.494622+0000","last_peered":"2024-04-25T17:08:13.494622+0000","last_clean":"2024-04-25T17:08:13.494622+0000","last_became_active":"2024-04-25T17:08:11.152573+0000","last_became_peered":"2024-04-25T17:08:11.152573+0000","last_unstale":"2024-04-25T17:08:13.494622+0000","last_undegraded":"2024-04-25T17:08:13.494622+0000","last_fullsized":"2024-04-25T17:08:13.494622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:13.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121535+0000","last_change":"2024-04-25T17:08:12.165939+0000","last_active":"2024-04-25T17:08:14.121535+0000","last_peered":"2024-04-25T17:08:14.121535+0000","last_clean":"2024-04-25T17:08:14.121535+0000","last_became_active":"2024-04-25T17:08:12.165569+0000","last_became_peered":"2024-04-25T17:08:12.165569+0000","last_unstale":"2024-04-25T17:08:14.121535+0000","last_undegraded":"2024-04-25T17:08:14.121535+0000","last_fullsized":"2024-04-25T17:08:14.121535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:15:16.230457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495142+0000","last_change":"2024-04-25T17:08:11.158927+0000","last_active":"2024-04-25T17:08:13.495142+0000","last_peered":"2024-04-25T17:08:13.495142+0000","last_clean":"2024-04-25T17:08:13.495142+0000","last_became_active":"2024-04-25T17:08:11.158528+0000","last_became_peered":"2024-04-25T17:08:11.158528+0000","last_unstale":"2024-04-25T17:08:13.495142+0000","last_undegraded":"2024-04-25T17:08:13.495142+0000","last_fullsized":"2024-04-25T17:08:13.495142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.308627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495691+0000","last_change":"2024-04-25T17:08:12.160776+0000","last_active":"2024-04-25T17:08:13.495691+0000","last_peered":"2024-04-25T17:08:13.495691+0000","last_clean":"2024-04-25T17:08:13.495691+0000","last_became_active":"2024-04-25T17:08:12.160409+0000","last_became_peered":"2024-04-25T17:08:12.160409+0000","last_unstale":"2024-04-25T17:08:13.495691+0000","last_undegraded":"2024-04-25T17:08:13.495691+0000","last_fullsized":"2024-04-25T17:08:13.495691+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:51.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495648+0000","last_change":"2024-04-25T17:08:11.153214+0000","last_active":"2024-04-25T17:08:13.495648+0000","last_peered":"2024-04-25T17:08:13.495648+0000","last_clean":"2024-04-25T17:08:13.495648+0000","last_became_active":"2024-04-25T17:08:11.152728+0000","last_became_peered":"2024-04-25T17:08:11.152728+0000","last_unstale":"2024-04-25T17:08:13.495648+0000","last_undegraded":"2024-04-25T17:08:13.495648+0000","last_fullsized":"2024-04-25T17:08:13.495648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.338549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141314+0000","last_change":"2024-04-25T17:08:12.162167+0000","last_active":"2024-04-25T17:08:13.141314+0000","last_peered":"2024-04-25T17:08:13.141314+0000","last_clean":"2024-04-25T17:08:13.141314+0000","last_became_active":"2024-04-25T17:08:12.160896+0000","last_became_peered":"2024-04-25T17:08:12.160896+0000","last_unstale":"2024-04-25T17:08:13.141314+0000","last_undegraded":"2024-04-25T17:08:13.141314+0000","last_fullsized":"2024-04-25T17:08:13.141314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:02.960832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495286+0000","last_change":"2024-04-25T17:08:11.158995+0000","last_active":"2024-04-25T17:08:13.495286+0000","last_peered":"2024-04-25T17:08:13.495286+0000","last_clean":"2024-04-25T17:08:13.495286+0000","last_became_active":"2024-04-25T17:08:11.158644+0000","last_became_peered":"2024-04-25T17:08:11.158644+0000","last_unstale":"2024-04-25T17:08:13.495286+0000","last_undegraded":"2024-04-25T17:08:13.495286+0000","last_fullsized":"2024-04-25T17:08:13.495286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:22.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141392+0000","last_change":"2024-04-25T17:08:12.161635+0000","last_active":"2024-04-25T17:08:13.141392+0000","last_peered":"2024-04-25T17:08:13.141392+0000","last_clean":"2024-04-25T17:08:13.141392+0000","last_became_active":"2024-04-25T17:08:12.153728+0000","last_became_peered":"2024-04-25T17:08:12.153728+0000","last_unstale":"2024-04-25T17:08:13.141392+0000","last_undegraded":"2024-04-25T17:08:13.141392+0000","last_fullsized":"2024-04-25T17:08:13.141392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:02.003550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121630+0000","last_change":"2024-04-25T17:08:11.150490+0000","last_active":"2024-04-25T17:08:14.121630+0000","last_peered":"2024-04-25T17:08:14.121630+0000","last_clean":"2024-04-25T17:08:14.121630+0000","last_became_active":"2024-04-25T17:08:11.148391+0000","last_became_peered":"2024-04-25T17:08:11.148391+0000","last_unstale":"2024-04-25T17:08:14.121630+0000","last_undegraded":"2024-04-25T17:08:14.121630+0000","last_fullsized":"2024-04-25T17:08:14.121630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:40.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141295+0000","last_change":"2024-04-25T17:08:12.168136+0000","last_active":"2024-04-25T17:08:13.141295+0000","last_peered":"2024-04-25T17:08:13.141295+0000","last_clean":"2024-04-25T17:08:13.141295+0000","last_became_active":"2024-04-25T17:08:12.167786+0000","last_became_peered":"2024-04-25T17:08:12.167786+0000","last_unstale":"2024-04-25T17:08:13.141295+0000","last_undegraded":"2024-04-25T17:08:13.141295+0000","last_fullsized":"2024-04-25T17:08:13.141295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:22.869503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495978+0000","last_change":"2024-04-25T17:08:12.159952+0000","last_active":"2024-04-25T17:08:13.495978+0000","last_peered":"2024-04-25T17:08:13.495978+0000","last_clean":"2024-04-25T17:08:13.495978+0000","last_became_active":"2024-04-25T17:08:12.159429+0000","last_became_peered":"2024-04-25T17:08:12.159429+0000","last_unstale":"2024-04-25T17:08:13.495978+0000","last_undegraded":"2024-04-25T17:08:13.495978+0000","last_fullsized":"2024-04-25T17:08:13.495978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:13:30.521084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495045+0000","last_change":"2024-04-25T17:08:11.149085+0000","last_active":"2024-04-25T17:08:13.495045+0000","last_peered":"2024-04-25T17:08:13.495045+0000","last_clean":"2024-04-25T17:08:13.495045+0000","last_became_active":"2024-04-25T17:08:11.148712+0000","last_became_peered":"2024-04-25T17:08:11.148712+0000","last_unstale":"2024-04-25T17:08:13.495045+0000","last_undegraded":"2024-04-25T17:08:13.495045+0000","last_fullsized":"2024-04-25T17:08:13.495045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:12.764034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140954+0000","last_change":"2024-04-25T17:08:12.178154+0000","last_active":"2024-04-25T17:08:13.140954+0000","last_peered":"2024-04-25T17:08:13.140954+0000","last_clean":"2024-04-25T17:08:13.140954+0000","last_became_active":"2024-04-25T17:08:12.177773+0000","last_became_peered":"2024-04-25T17:08:12.177773+0000","last_unstale":"2024-04-25T17:08:13.140954+0000","last_undegraded":"2024-04-25T17:08:13.140954+0000","last_fullsized":"2024-04-25T17:08:13.140954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.962410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494975+0000","last_change":"2024-04-25T17:08:11.152932+0000","last_active":"2024-04-25T17:08:13.494975+0000","last_peered":"2024-04-25T17:08:13.494975+0000","last_clean":"2024-04-25T17:08:13.494975+0000","last_became_active":"2024-04-25T17:08:11.152530+0000","last_became_peered":"2024-04-25T17:08:11.152530+0000","last_unstale":"2024-04-25T17:08:13.494975+0000","last_undegraded":"2024-04-25T17:08:13.494975+0000","last_fullsized":"2024-04-25T17:08:13.494975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121425+0000","last_change":"2024-04-25T17:06:55.951326+0000","last_active":"2024-04-25T17:08:14.121425+0000","last_peered":"2024-04-25T17:08:14.121425+0000","last_clean":"2024-04-25T17:08:14.121425+0000","last_became_active":"2024-04-25T17:06:55.946641+0000","last_became_peered":"2024-04-25T17:06:55.946641+0000","last_unstale":"2024-04-25T17:08:14.121425+0000","last_undegraded":"2024-04-25T17:08:14.121425+0000","last_fullsized":"2024-04-25T17:08:14.121425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_clean_scrub_stamp":"2024-04-25T17:03:27.394276+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:58.308843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494922+0000","last_change":"2024-04-25T17:08:12.162065+0000","last_active":"2024-04-25T17:08:13.494922+0000","last_peered":"2024-04-25T17:08:13.494922+0000","last_clean":"2024-04-25T17:08:13.494922+0000","last_became_active":"2024-04-25T17:08:12.161733+0000","last_became_peered":"2024-04-25T17:08:12.161733+0000","last_unstale":"2024-04-25T17:08:13.494922+0000","last_undegraded":"2024-04-25T17:08:13.494922+0000","last_fullsized":"2024-04-25T17:08:13.494922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:28.870514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121612+0000","last_change":"2024-04-25T17:08:11.155941+0000","last_active":"2024-04-25T17:08:14.121612+0000","last_peered":"2024-04-25T17:08:14.121612+0000","last_clean":"2024-04-25T17:08:14.121612+0000","last_became_active":"2024-04-25T17:08:11.155381+0000","last_became_peered":"2024-04-25T17:08:11.155381+0000","last_unstale":"2024-04-25T17:08:14.121612+0000","last_undegraded":"2024-04-25T17:08:14.121612+0000","last_fullsized":"2024-04-25T17:08:14.121612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:28.423462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496103+0000","last_change":"2024-04-25T17:08:11.158960+0000","last_active":"2024-04-25T17:08:13.496103+0000","last_peered":"2024-04-25T17:08:13.496103+0000","last_clean":"2024-04-25T17:08:13.496103+0000","last_became_active":"2024-04-25T17:08:11.158609+0000","last_became_peered":"2024-04-25T17:08:11.158609+0000","last_unstale":"2024-04-25T17:08:13.496103+0000","last_undegraded":"2024-04-25T17:08:13.496103+0000","last_fullsized":"2024-04-25T17:08:13.496103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:19.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495147+0000","last_change":"2024-04-25T17:08:12.161483+0000","last_active":"2024-04-25T17:08:13.495147+0000","last_peered":"2024-04-25T17:08:13.495147+0000","last_clean":"2024-04-25T17:08:13.495147+0000","last_became_active":"2024-04-25T17:08:12.158586+0000","last_became_peered":"2024-04-25T17:08:12.158586+0000","last_unstale":"2024-04-25T17:08:13.495147+0000","last_undegraded":"2024-04-25T17:08:13.495147+0000","last_fullsized":"2024-04-25T17:08:13.495147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.551328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'3","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:16.146220+0000","last_change":"2024-04-25T17:08:11.161497+0000","last_active":"2024-04-25T17:08:16.146220+0000","last_peered":"2024-04-25T17:08:16.146220+0000","last_clean":"2024-04-25T17:08:16.146220+0000","last_became_active":"2024-04-25T17:08:11.160990+0000","last_became_peered":"2024-04-25T17:08:11.160990+0000","last_unstale":"2024-04-25T17:08:16.146220+0000","last_undegraded":"2024-04-25T17:08:16.146220+0000","last_fullsized":"2024-04-25T17:08:16.146220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:59.682133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141148+0000","last_change":"2024-04-25T17:08:12.171214+0000","last_active":"2024-04-25T17:08:13.141148+0000","last_peered":"2024-04-25T17:08:13.141148+0000","last_clean":"2024-04-25T17:08:13.141148+0000","last_became_active":"2024-04-25T17:08:12.170877+0000","last_became_peered":"2024-04-25T17:08:12.170877+0000","last_unstale":"2024-04-25T17:08:13.141148+0000","last_undegraded":"2024-04-25T17:08:13.141148+0000","last_fullsized":"2024-04-25T17:08:13.141148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:18.442691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494978+0000","last_change":"2024-04-25T17:08:11.154819+0000","last_active":"2024-04-25T17:08:13.494978+0000","last_peered":"2024-04-25T17:08:13.494978+0000","last_clean":"2024-04-25T17:08:13.494978+0000","last_became_active":"2024-04-25T17:08:11.154006+0000","last_became_peered":"2024-04-25T17:08:11.154006+0000","last_unstale":"2024-04-25T17:08:13.494978+0000","last_undegraded":"2024-04-25T17:08:13.494978+0000","last_fullsized":"2024-04-25T17:08:13.494978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.063058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422056+0000","last_change":"2024-04-25T17:08:12.167301+0000","last_active":"2024-04-25T17:08:13.422056+0000","last_peered":"2024-04-25T17:08:13.422056+0000","last_clean":"2024-04-25T17:08:13.422056+0000","last_became_active":"2024-04-25T17:08:12.166944+0000","last_became_peered":"2024-04-25T17:08:12.166944+0000","last_unstale":"2024-04-25T17:08:13.422056+0000","last_undegraded":"2024-04-25T17:08:13.422056+0000","last_fullsized":"2024-04-25T17:08:13.422056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:46.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495108+0000","last_change":"2024-04-25T17:08:11.156616+0000","last_active":"2024-04-25T17:08:13.495108+0000","last_peered":"2024-04-25T17:08:13.495108+0000","last_clean":"2024-04-25T17:08:13.495108+0000","last_became_active":"2024-04-25T17:08:11.156256+0000","last_became_peered":"2024-04-25T17:08:11.156256+0000","last_unstale":"2024-04-25T17:08:13.495108+0000","last_undegraded":"2024-04-25T17:08:13.495108+0000","last_fullsized":"2024-04-25T17:08:13.495108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:37.539773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495319+0000","last_change":"2024-04-25T17:08:12.175216+0000","last_active":"2024-04-25T17:08:13.495319+0000","last_peered":"2024-04-25T17:08:13.495319+0000","last_clean":"2024-04-25T17:08:13.495319+0000","last_became_active":"2024-04-25T17:08:12.174521+0000","last_became_peered":"2024-04-25T17:08:12.174521+0000","last_unstale":"2024-04-25T17:08:13.495319+0000","last_undegraded":"2024-04-25T17:08:13.495319+0000","last_fullsized":"2024-04-25T17:08:13.495319+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.322733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140606+0000","last_change":"2024-04-25T17:08:11.162287+0000","last_active":"2024-04-25T17:08:13.140606+0000","last_peered":"2024-04-25T17:08:13.140606+0000","last_clean":"2024-04-25T17:08:13.140606+0000","last_became_active":"2024-04-25T17:08:11.158297+0000","last_became_peered":"2024-04-25T17:08:11.158297+0000","last_unstale":"2024-04-25T17:08:13.140606+0000","last_undegraded":"2024-04-25T17:08:13.140606+0000","last_fullsized":"2024-04-25T17:08:13.140606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:15.758880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141260+0000","last_change":"2024-04-25T17:08:12.163535+0000","last_active":"2024-04-25T17:08:13.141260+0000","last_peered":"2024-04-25T17:08:13.141260+0000","last_clean":"2024-04-25T17:08:13.141260+0000","last_became_active":"2024-04-25T17:08:12.162848+0000","last_became_peered":"2024-04-25T17:08:12.162848+0000","last_unstale":"2024-04-25T17:08:13.141260+0000","last_undegraded":"2024-04-25T17:08:13.141260+0000","last_fullsized":"2024-04-25T17:08:13.141260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:50.288869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352187+0000","last_change":"2024-04-25T17:08:11.162780+0000","last_active":"2024-04-25T17:08:13.352187+0000","last_peered":"2024-04-25T17:08:13.352187+0000","last_clean":"2024-04-25T17:08:13.352187+0000","last_became_active":"2024-04-25T17:08:11.162385+0000","last_became_peered":"2024-04-25T17:08:11.162385+0000","last_unstale":"2024-04-25T17:08:13.352187+0000","last_undegraded":"2024-04-25T17:08:13.352187+0000","last_fullsized":"2024-04-25T17:08:13.352187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:25.420740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140869+0000","last_change":"2024-04-25T17:08:12.170828+0000","last_active":"2024-04-25T17:08:13.140869+0000","last_peered":"2024-04-25T17:08:13.140869+0000","last_clean":"2024-04-25T17:08:13.140869+0000","last_became_active":"2024-04-25T17:08:12.170479+0000","last_became_peered":"2024-04-25T17:08:12.170479+0000","last_unstale":"2024-04-25T17:08:13.140869+0000","last_undegraded":"2024-04-25T17:08:13.140869+0000","last_fullsized":"2024-04-25T17:08:13.140869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.947118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495662+0000","last_change":"2024-04-25T17:08:11.156754+0000","last_active":"2024-04-25T17:08:13.495662+0000","last_peered":"2024-04-25T17:08:13.495662+0000","last_clean":"2024-04-25T17:08:13.495662+0000","last_became_active":"2024-04-25T17:08:11.156414+0000","last_became_peered":"2024-04-25T17:08:11.156414+0000","last_unstale":"2024-04-25T17:08:13.495662+0000","last_undegraded":"2024-04-25T17:08:13.495662+0000","last_fullsized":"2024-04-25T17:08:13.495662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:10.364234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121978+0000","last_change":"2024-04-25T17:08:12.162032+0000","last_active":"2024-04-25T17:08:14.121978+0000","last_peered":"2024-04-25T17:08:14.121978+0000","last_clean":"2024-04-25T17:08:14.121978+0000","last_became_active":"2024-04-25T17:08:12.161017+0000","last_became_peered":"2024-04-25T17:08:12.161017+0000","last_unstale":"2024-04-25T17:08:14.121978+0000","last_undegraded":"2024-04-25T17:08:14.121978+0000","last_fullsized":"2024-04-25T17:08:14.121978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.472984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352660+0000","last_change":"2024-04-25T17:08:11.149550+0000","last_active":"2024-04-25T17:08:13.352660+0000","last_peered":"2024-04-25T17:08:13.352660+0000","last_clean":"2024-04-25T17:08:13.352660+0000","last_became_active":"2024-04-25T17:08:11.149082+0000","last_became_peered":"2024-04-25T17:08:11.149082+0000","last_unstale":"2024-04-25T17:08:13.352660+0000","last_undegraded":"2024-04-25T17:08:13.352660+0000","last_fullsized":"2024-04-25T17:08:13.352660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:14.565877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122122+0000","last_change":"2024-04-25T17:08:12.160331+0000","last_active":"2024-04-25T17:08:14.122122+0000","last_peered":"2024-04-25T17:08:14.122122+0000","last_clean":"2024-04-25T17:08:14.122122+0000","last_became_active":"2024-04-25T17:08:12.159931+0000","last_became_peered":"2024-04-25T17:08:12.159931+0000","last_unstale":"2024-04-25T17:08:14.122122+0000","last_undegraded":"2024-04-25T17:08:14.122122+0000","last_fullsized":"2024-04-25T17:08:14.122122+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:37.246974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496227+0000","last_change":"2024-04-25T17:08:11.163118+0000","last_active":"2024-04-25T17:08:13.496227+0000","last_peered":"2024-04-25T17:08:13.496227+0000","last_clean":"2024-04-25T17:08:13.496227+0000","last_became_active":"2024-04-25T17:08:11.161691+0000","last_became_peered":"2024-04-25T17:08:11.161691+0000","last_unstale":"2024-04-25T17:08:13.496227+0000","last_undegraded":"2024-04-25T17:08:13.496227+0000","last_fullsized":"2024-04-25T17:08:13.496227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:32.006098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121927+0000","last_change":"2024-04-25T17:08:12.168387+0000","last_active":"2024-04-25T17:08:14.121927+0000","last_peered":"2024-04-25T17:08:14.121927+0000","last_clean":"2024-04-25T17:08:14.121927+0000","last_became_active":"2024-04-25T17:08:12.168008+0000","last_became_peered":"2024-04-25T17:08:12.168008+0000","last_unstale":"2024-04-25T17:08:14.121927+0000","last_undegraded":"2024-04-25T17:08:14.121927+0000","last_fullsized":"2024-04-25T17:08:14.121927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:08.438945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495492+0000","last_change":"2024-04-25T17:08:11.149181+0000","last_active":"2024-04-25T17:08:13.495492+0000","last_peered":"2024-04-25T17:08:13.495492+0000","last_clean":"2024-04-25T17:08:13.495492+0000","last_became_active":"2024-04-25T17:08:11.148847+0000","last_became_peered":"2024-04-25T17:08:11.148847+0000","last_unstale":"2024-04-25T17:08:13.495492+0000","last_undegraded":"2024-04-25T17:08:13.495492+0000","last_fullsized":"2024-04-25T17:08:13.495492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:40.720578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122115+0000","last_change":"2024-04-25T17:08:12.162173+0000","last_active":"2024-04-25T17:08:14.122115+0000","last_peered":"2024-04-25T17:08:14.122115+0000","last_clean":"2024-04-25T17:08:14.122115+0000","last_became_active":"2024-04-25T17:08:12.161818+0000","last_became_peered":"2024-04-25T17:08:12.161818+0000","last_unstale":"2024-04-25T17:08:14.122115+0000","last_undegraded":"2024-04-25T17:08:14.122115+0000","last_fullsized":"2024-04-25T17:08:14.122115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:28.872937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141351+0000","last_change":"2024-04-25T17:08:12.165542+0000","last_active":"2024-04-25T17:08:13.141351+0000","last_peered":"2024-04-25T17:08:13.141351+0000","last_clean":"2024-04-25T17:08:13.141351+0000","last_became_active":"2024-04-25T17:08:12.165190+0000","last_became_peered":"2024-04-25T17:08:12.165190+0000","last_unstale":"2024-04-25T17:08:13.141351+0000","last_undegraded":"2024-04-25T17:08:13.141351+0000","last_fullsized":"2024-04-25T17:08:13.141351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:51.650953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352355+0000","last_change":"2024-04-25T17:08:12.161429+0000","last_active":"2024-04-25T17:08:13.352355+0000","last_peered":"2024-04-25T17:08:13.352355+0000","last_clean":"2024-04-25T17:08:13.352355+0000","last_became_active":"2024-04-25T17:08:12.159184+0000","last_became_peered":"2024-04-25T17:08:12.159184+0000","last_unstale":"2024-04-25T17:08:13.352355+0000","last_undegraded":"2024-04-25T17:08:13.352355+0000","last_fullsized":"2024-04-25T17:08:13.352355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:56:21.366775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352298+0000","last_change":"2024-04-25T17:08:12.169948+0000","last_active":"2024-04-25T17:08:13.352298+0000","last_peered":"2024-04-25T17:08:13.352298+0000","last_clean":"2024-04-25T17:08:13.352298+0000","last_became_active":"2024-04-25T17:08:12.169592+0000","last_became_peered":"2024-04-25T17:08:12.169592+0000","last_unstale":"2024-04-25T17:08:13.352298+0000","last_undegraded":"2024-04-25T17:08:13.352298+0000","last_fullsized":"2024-04-25T17:08:13.352298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:22.749954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421852+0000","last_change":"2024-04-25T17:08:12.169785+0000","last_active":"2024-04-25T17:08:13.421852+0000","last_peered":"2024-04-25T17:08:13.421852+0000","last_clean":"2024-04-25T17:08:13.421852+0000","last_became_active":"2024-04-25T17:08:12.169226+0000","last_became_peered":"2024-04-25T17:08:12.169226+0000","last_unstale":"2024-04-25T17:08:13.421852+0000","last_undegraded":"2024-04-25T17:08:13.421852+0000","last_fullsized":"2024-04-25T17:08:13.421852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.232112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495028+0000","last_change":"2024-04-25T17:08:12.157992+0000","last_active":"2024-04-25T17:08:13.495028+0000","last_peered":"2024-04-25T17:08:13.495028+0000","last_clean":"2024-04-25T17:08:13.495028+0000","last_became_active":"2024-04-25T17:08:12.157636+0000","last_became_peered":"2024-04-25T17:08:12.157636+0000","last_unstale":"2024-04-25T17:08:13.495028+0000","last_undegraded":"2024-04-25T17:08:13.495028+0000","last_fullsized":"2024-04-25T17:08:13.495028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:50.530074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141353+0000","last_change":"2024-04-25T17:08:12.160977+0000","last_active":"2024-04-25T17:08:13.141353+0000","last_peered":"2024-04-25T17:08:13.141353+0000","last_clean":"2024-04-25T17:08:13.141353+0000","last_became_active":"2024-04-25T17:08:12.160571+0000","last_became_peered":"2024-04-25T17:08:12.160571+0000","last_unstale":"2024-04-25T17:08:13.141353+0000","last_undegraded":"2024-04-25T17:08:13.141353+0000","last_fullsized":"2024-04-25T17:08:13.141353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.882628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352838+0000","last_change":"2024-04-25T17:08:12.158566+0000","last_active":"2024-04-25T17:08:13.352838+0000","last_peered":"2024-04-25T17:08:13.352838+0000","last_clean":"2024-04-25T17:08:13.352838+0000","last_became_active":"2024-04-25T17:08:12.157911+0000","last_became_peered":"2024-04-25T17:08:12.157911+0000","last_unstale":"2024-04-25T17:08:13.352838+0000","last_undegraded":"2024-04-25T17:08:13.352838+0000","last_fullsized":"2024-04-25T17:08:13.352838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:49.665779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141146+0000","last_change":"2024-04-25T17:08:12.174931+0000","last_active":"2024-04-25T17:08:13.141146+0000","last_peered":"2024-04-25T17:08:13.141146+0000","last_clean":"2024-04-25T17:08:13.141146+0000","last_became_active":"2024-04-25T17:08:12.174571+0000","last_became_peered":"2024-04-25T17:08:12.174571+0000","last_unstale":"2024-04-25T17:08:13.141146+0000","last_undegraded":"2024-04-25T17:08:13.141146+0000","last_fullsized":"2024-04-25T17:08:13.141146+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:55.832578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495557+0000","last_change":"2024-04-25T17:08:12.165592+0000","last_active":"2024-04-25T17:08:13.495557+0000","last_peered":"2024-04-25T17:08:13.495557+0000","last_clean":"2024-04-25T17:08:13.495557+0000","last_became_active":"2024-04-25T17:08:12.165273+0000","last_became_peered":"2024-04-25T17:08:12.165273+0000","last_unstale":"2024-04-25T17:08:13.495557+0000","last_undegraded":"2024-04-25T17:08:13.495557+0000","last_fullsized":"2024-04-25T17:08:13.495557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:25.199657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140590+0000","last_change":"2024-04-25T17:08:12.174095+0000","last_active":"2024-04-25T17:08:13.140590+0000","last_peered":"2024-04-25T17:08:13.140590+0000","last_clean":"2024-04-25T17:08:13.140590+0000","last_became_active":"2024-04-25T17:08:12.172786+0000","last_became_peered":"2024-04-25T17:08:12.172786+0000","last_unstale":"2024-04-25T17:08:13.140590+0000","last_undegraded":"2024-04-25T17:08:13.140590+0000","last_fullsized":"2024-04-25T17:08:13.140590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.736300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495785+0000","last_change":"2024-04-25T17:08:12.158772+0000","last_active":"2024-04-25T17:08:13.495785+0000","last_peered":"2024-04-25T17:08:13.495785+0000","last_clean":"2024-04-25T17:08:13.495785+0000","last_became_active":"2024-04-25T17:08:12.158417+0000","last_became_peered":"2024-04-25T17:08:12.158417+0000","last_unstale":"2024-04-25T17:08:13.495785+0000","last_undegraded":"2024-04-25T17:08:13.495785+0000","last_fullsized":"2024-04-25T17:08:13.495785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:21.837887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495613+0000","last_change":"2024-04-25T17:08:12.162384+0000","last_active":"2024-04-25T17:08:13.495613+0000","last_peered":"2024-04-25T17:08:13.495613+0000","last_clean":"2024-04-25T17:08:13.495613+0000","last_became_active":"2024-04-25T17:08:12.161997+0000","last_became_peered":"2024-04-25T17:08:12.161997+0000","last_unstale":"2024-04-25T17:08:13.495613+0000","last_undegraded":"2024-04-25T17:08:13.495613+0000","last_fullsized":"2024-04-25T17:08:13.495613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:17.358171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141217+0000","last_change":"2024-04-25T17:08:12.159015+0000","last_active":"2024-04-25T17:08:13.141217+0000","last_peered":"2024-04-25T17:08:13.141217+0000","last_clean":"2024-04-25T17:08:13.141217+0000","last_became_active":"2024-04-25T17:08:12.157999+0000","last_became_peered":"2024-04-25T17:08:12.157999+0000","last_unstale":"2024-04-25T17:08:13.141217+0000","last_undegraded":"2024-04-25T17:08:13.141217+0000","last_fullsized":"2024-04-25T17:08:13.141217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:00.291221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280965,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":49,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":80392,"data_compressed":3686,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678029,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":389120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874258,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":981294,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037785,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":815104,"data_stored":389702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234013,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":819200,"data_stored":389724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":864256,"data_stored":980070,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":815104,"data_stored":389237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":427313,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":815104,"data_stored":389210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":410049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":823296,"data_stored":389341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":815104,"data_stored":389142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":979670,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38193,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38227,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T17:08:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:19 smithi097 ceph-mon[32602]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:19 smithi097 ceph-mon[32602]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:19 smithi097 ceph-mon[32602]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:19 smithi097 ceph-mon[32602]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 776 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T17:08:19.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:19 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1869148905' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-25T17:08:19.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:19 smithi171 ceph-mon[33189]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:19 smithi171 ceph-mon[33189]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:19 smithi171 ceph-mon[33189]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:19.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:19 smithi171 ceph-mon[33189]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 776 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T17:08:19.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:19 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1869148905' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-25T17:08:20.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:20 smithi096 ceph-mon[29131]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:20.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:20 smithi097 ceph-mon[32602]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:20.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:20 smithi171 ceph-mon[33189]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:21 smithi096 ceph-mon[29131]: pgmap v285: 97 pgs: 97 active+clean; 615 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-04-25T17:08:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:21 smithi097 ceph-mon[32602]: pgmap v285: 97 pgs: 97 active+clean; 615 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-04-25T17:08:21.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:21 smithi171 ceph-mon[33189]: pgmap v285: 97 pgs: 97 active+clean; 615 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-04-25T17:08:22.654 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T17:08:22.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3932090876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8102250 msgr2=0x7f52d810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:22.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3932090876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8102250 0x7f52d810bcb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009900 tx=0x7f52d002a410 comp rx=0 tx=0).stop 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3932090876 shutdown_connections 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3932090876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8102250 0x7f52d810bcb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3932090876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52d8101750 0x7f52d8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3932090876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d81050a0 0x7f52d81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3932090876 >> 172.21.15.96:0/3932090876 conn(0x7f52d80fd370 msgr2=0x7f52d80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3932090876 shutdown_connections 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.778+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3932090876 wait complete. 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 Processor -- start 2024-04-25T17:08:22.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 -- start start 2024-04-25T17:08:22.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8101750 0x7f52d819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:22.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52d8102250 0x7f52d819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:22.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d81050a0 0x7f52d81a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52d810f630 con 0x7f52d8101750 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f52d810f4b0 con 0x7f52d8102250 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52ddaf0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52d810f7b0 con 0x7f52d81050a0 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8101750 0x7f52d819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8101750 0x7f52d819a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47712/0 (socket says 172.21.15.96:47712) 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52d77fe640 1 -- 172.21.15.96:0/3978407264 learned_addr learned my addr 172.21.15.96:0/3978407264 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.779+0000 7f52d7fff640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d81050a0 0x7f52d81a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d6ffd640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52d8102250 0x7f52d819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d77fe640 1 -- 172.21.15.96:0/3978407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d81050a0 msgr2=0x7f52d81a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d77fe640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d81050a0 0x7f52d81a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d77fe640 1 -- 172.21.15.96:0/3978407264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52d8102250 msgr2=0x7f52d819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d77fe640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52d8102250 0x7f52d819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d77fe640 1 -- 172.21.15.96:0/3978407264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81a1c00 con 0x7f52d8101750 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d7fff640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d81050a0 0x7f52d81a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d6ffd640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52d8102250 0x7f52d819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d77fe640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8101750 0x7f52d819a170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ead0 tx=0x7f52c800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d4ff9640 1 -- 172.21.15.96:0/3978407264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c800cde0 con 0x7f52d8101750 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d81a1ef0 con 0x7f52d8101750 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d4ff9640 1 -- 172.21.15.96:0/3978407264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c8003e20 con 0x7f52d8101750 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52d4ff9640 1 -- 172.21.15.96:0/3978407264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80116d0 con 0x7f52d8101750 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.780+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81a2350 con 0x7f52d8101750 2024-04-25T17:08:22.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.781+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d806a190 con 0x7f52d8101750 2024-04-25T17:08:22.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.783+0000 7f52d4ff9640 1 -- 172.21.15.96:0/3978407264 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f52c8004270 con 0x7f52d8101750 2024-04-25T17:08:22.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.783+0000 7f52d4ff9640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52c00afb80 0x7f52c00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:22.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.783+0000 7f52d6ffd640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52c00afb80 0x7f52c00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:22.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.783+0000 7f52d4ff9640 1 -- 172.21.15.96:0/3978407264 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f52c80df300 con 0x7f52d8101750 2024-04-25T17:08:22.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.784+0000 7f52d6ffd640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52c00afb80 0x7f52c00b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52d819b210 tx=0x7f52cc005db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:22.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.787+0000 7f52d4ff9640 1 -- 172.21.15.96:0/3978407264 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52c8015030 con 0x7f52d8101750 2024-04-25T17:08:22.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.961+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52d8062fe0 con 0x7f52c00afb80 2024-04-25T17:08:22.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.968+0000 7f52d4ff9640 1 -- 172.21.15.96:0/3978407264 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258866 (secure 0 0 0) 0x7f52d8062fe0 con 0x7f52c00afb80 2024-04-25T17:08:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:22.971 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T17:08:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52c00afb80 msgr2=0x7f52c00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52c00afb80 0x7f52c00b2090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52d819b210 tx=0x7f52cc005db0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8101750 msgr2=0x7f52d819a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8101750 0x7f52d819a170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ead0 tx=0x7f52c800ef90 comp rx=0 tx=0).stop 2024-04-25T17:08:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 shutdown_connections 2024-04-25T17:08:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f52c00afb80 0x7f52c00b2090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d81050a0 0x7f52d81a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52d8102250 0x7f52d819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 --2- 172.21.15.96:0/3978407264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52d8101750 0x7f52d819a170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 >> 172.21.15.96:0/3978407264 conn(0x7f52d80fd370 msgr2=0x7f52d8107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 shutdown_connections 2024-04-25T17:08:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:22.971+0000 7f52ddaf0640 1 -- 172.21.15.96:0/3978407264 wait complete. 2024-04-25T17:08:22.982 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-04-25T17:08:21.051326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629593,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":101,"num_read_kb":118,"num_write":185,"num_write_kb":2206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359016,"kb_used_data":9840,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582808,"statfs":{"total":1151940427776,"available":1151572795392,"internally_reserved":0,"allocated":10076160,"data_stored":6540687,"data_compressed":110876,"data_compressed_allocated":143360,"data_compressed_original":1785856,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39225,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":20,"num_read_kb":49,"num_write":62,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.911353"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141585+0000","last_change":"2024-04-25T17:08:12.162430+0000","last_active":"2024-04-25T17:08:13.141585+0000","last_peered":"2024-04-25T17:08:13.141585+0000","last_clean":"2024-04-25T17:08:13.141585+0000","last_became_active":"2024-04-25T17:08:12.162052+0000","last_became_peered":"2024-04-25T17:08:12.162052+0000","last_unstale":"2024-04-25T17:08:13.141585+0000","last_undegraded":"2024-04-25T17:08:13.141585+0000","last_fullsized":"2024-04-25T17:08:13.141585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:09.637937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421957+0000","last_change":"2024-04-25T17:08:12.169139+0000","last_active":"2024-04-25T17:08:13.421957+0000","last_peered":"2024-04-25T17:08:13.421957+0000","last_clean":"2024-04-25T17:08:13.421957+0000","last_became_active":"2024-04-25T17:08:12.168715+0000","last_became_peered":"2024-04-25T17:08:12.168715+0000","last_unstale":"2024-04-25T17:08:13.421957+0000","last_undegraded":"2024-04-25T17:08:13.421957+0000","last_fullsized":"2024-04-25T17:08:13.421957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:57.357750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421733+0000","last_change":"2024-04-25T17:08:12.159141+0000","last_active":"2024-04-25T17:08:13.421733+0000","last_peered":"2024-04-25T17:08:13.421733+0000","last_clean":"2024-04-25T17:08:13.421733+0000","last_became_active":"2024-04-25T17:08:12.158796+0000","last_became_peered":"2024-04-25T17:08:12.158796+0000","last_unstale":"2024-04-25T17:08:13.421733+0000","last_undegraded":"2024-04-25T17:08:13.421733+0000","last_fullsized":"2024-04-25T17:08:13.421733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:55.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495311+0000","last_change":"2024-04-25T17:08:12.160464+0000","last_active":"2024-04-25T17:08:13.495311+0000","last_peered":"2024-04-25T17:08:13.495311+0000","last_clean":"2024-04-25T17:08:13.495311+0000","last_became_active":"2024-04-25T17:08:12.160118+0000","last_became_peered":"2024-04-25T17:08:12.160118+0000","last_unstale":"2024-04-25T17:08:13.495311+0000","last_undegraded":"2024-04-25T17:08:13.495311+0000","last_fullsized":"2024-04-25T17:08:13.495311+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:59:06.465964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495515+0000","last_change":"2024-04-25T17:08:12.174591+0000","last_active":"2024-04-25T17:08:13.495515+0000","last_peered":"2024-04-25T17:08:13.495515+0000","last_clean":"2024-04-25T17:08:13.495515+0000","last_became_active":"2024-04-25T17:08:12.174173+0000","last_became_peered":"2024-04-25T17:08:12.174173+0000","last_unstale":"2024-04-25T17:08:13.495515+0000","last_undegraded":"2024-04-25T17:08:13.495515+0000","last_fullsized":"2024-04-25T17:08:13.495515+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:12.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141095+0000","last_change":"2024-04-25T17:08:12.164230+0000","last_active":"2024-04-25T17:08:13.141095+0000","last_peered":"2024-04-25T17:08:13.141095+0000","last_clean":"2024-04-25T17:08:13.141095+0000","last_became_active":"2024-04-25T17:08:12.163868+0000","last_became_peered":"2024-04-25T17:08:12.163868+0000","last_unstale":"2024-04-25T17:08:13.141095+0000","last_undegraded":"2024-04-25T17:08:13.141095+0000","last_fullsized":"2024-04-25T17:08:13.141095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:12.604164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121611+0000","last_change":"2024-04-25T17:08:12.161627+0000","last_active":"2024-04-25T17:08:14.121611+0000","last_peered":"2024-04-25T17:08:14.121611+0000","last_clean":"2024-04-25T17:08:14.121611+0000","last_became_active":"2024-04-25T17:08:12.161146+0000","last_became_peered":"2024-04-25T17:08:12.161146+0000","last_unstale":"2024-04-25T17:08:14.121611+0000","last_undegraded":"2024-04-25T17:08:14.121611+0000","last_fullsized":"2024-04-25T17:08:14.121611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.972053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140558+0000","last_change":"2024-04-25T17:08:12.165512+0000","last_active":"2024-04-25T17:08:13.140558+0000","last_peered":"2024-04-25T17:08:13.140558+0000","last_clean":"2024-04-25T17:08:13.140558+0000","last_became_active":"2024-04-25T17:08:12.164018+0000","last_became_peered":"2024-04-25T17:08:12.164018+0000","last_unstale":"2024-04-25T17:08:13.140558+0000","last_undegraded":"2024-04-25T17:08:13.140558+0000","last_fullsized":"2024-04-25T17:08:13.140558+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:28.075786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495771+0000","last_change":"2024-04-25T17:08:12.173313+0000","last_active":"2024-04-25T17:08:13.495771+0000","last_peered":"2024-04-25T17:08:13.495771+0000","last_clean":"2024-04-25T17:08:13.495771+0000","last_became_active":"2024-04-25T17:08:12.171269+0000","last_became_peered":"2024-04-25T17:08:12.171269+0000","last_unstale":"2024-04-25T17:08:13.495771+0000","last_undegraded":"2024-04-25T17:08:13.495771+0000","last_fullsized":"2024-04-25T17:08:13.495771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:39.565973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141559+0000","last_change":"2024-04-25T17:08:12.166114+0000","last_active":"2024-04-25T17:08:13.141559+0000","last_peered":"2024-04-25T17:08:13.141559+0000","last_clean":"2024-04-25T17:08:13.141559+0000","last_became_active":"2024-04-25T17:08:12.165737+0000","last_became_peered":"2024-04-25T17:08:12.165737+0000","last_unstale":"2024-04-25T17:08:13.141559+0000","last_undegraded":"2024-04-25T17:08:13.141559+0000","last_fullsized":"2024-04-25T17:08:13.141559+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:55.624239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122116+0000","last_change":"2024-04-25T17:08:12.160435+0000","last_active":"2024-04-25T17:08:14.122116+0000","last_peered":"2024-04-25T17:08:14.122116+0000","last_clean":"2024-04-25T17:08:14.122116+0000","last_became_active":"2024-04-25T17:08:12.160085+0000","last_became_peered":"2024-04-25T17:08:12.160085+0000","last_unstale":"2024-04-25T17:08:14.122116+0000","last_undegraded":"2024-04-25T17:08:14.122116+0000","last_fullsized":"2024-04-25T17:08:14.122116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.709567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495078+0000","last_change":"2024-04-25T17:08:12.173990+0000","last_active":"2024-04-25T17:08:13.495078+0000","last_peered":"2024-04-25T17:08:13.495078+0000","last_clean":"2024-04-25T17:08:13.495078+0000","last_became_active":"2024-04-25T17:08:12.172838+0000","last_became_peered":"2024-04-25T17:08:12.172838+0000","last_unstale":"2024-04-25T17:08:13.495078+0000","last_undegraded":"2024-04-25T17:08:13.495078+0000","last_fullsized":"2024-04-25T17:08:13.495078+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:00:56.720758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495615+0000","last_change":"2024-04-25T17:08:12.167579+0000","last_active":"2024-04-25T17:08:13.495615+0000","last_peered":"2024-04-25T17:08:13.495615+0000","last_clean":"2024-04-25T17:08:13.495615+0000","last_became_active":"2024-04-25T17:08:12.166583+0000","last_became_peered":"2024-04-25T17:08:12.166583+0000","last_unstale":"2024-04-25T17:08:13.495615+0000","last_undegraded":"2024-04-25T17:08:13.495615+0000","last_fullsized":"2024-04-25T17:08:13.495615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:44.081814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421592+0000","last_change":"2024-04-25T17:08:12.161263+0000","last_active":"2024-04-25T17:08:13.421592+0000","last_peered":"2024-04-25T17:08:13.421592+0000","last_clean":"2024-04-25T17:08:13.421592+0000","last_became_active":"2024-04-25T17:08:12.160078+0000","last_became_peered":"2024-04-25T17:08:12.160078+0000","last_unstale":"2024-04-25T17:08:13.421592+0000","last_undegraded":"2024-04-25T17:08:13.421592+0000","last_fullsized":"2024-04-25T17:08:13.421592+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:56.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421594+0000","last_change":"2024-04-25T17:08:12.167318+0000","last_active":"2024-04-25T17:08:13.421594+0000","last_peered":"2024-04-25T17:08:13.421594+0000","last_clean":"2024-04-25T17:08:13.421594+0000","last_became_active":"2024-04-25T17:08:12.166964+0000","last_became_peered":"2024-04-25T17:08:12.166964+0000","last_unstale":"2024-04-25T17:08:13.421594+0000","last_undegraded":"2024-04-25T17:08:13.421594+0000","last_fullsized":"2024-04-25T17:08:13.421594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:25.967818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494787+0000","last_change":"2024-04-25T17:08:12.166417+0000","last_active":"2024-04-25T17:08:13.494787+0000","last_peered":"2024-04-25T17:08:13.494787+0000","last_clean":"2024-04-25T17:08:13.494787+0000","last_became_active":"2024-04-25T17:08:12.165928+0000","last_became_peered":"2024-04-25T17:08:12.165928+0000","last_unstale":"2024-04-25T17:08:13.494787+0000","last_undegraded":"2024-04-25T17:08:13.494787+0000","last_fullsized":"2024-04-25T17:08:13.494787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:24:22.359814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141635+0000","last_change":"2024-04-25T17:08:12.164952+0000","last_active":"2024-04-25T17:08:13.141635+0000","last_peered":"2024-04-25T17:08:13.141635+0000","last_clean":"2024-04-25T17:08:13.141635+0000","last_became_active":"2024-04-25T17:08:12.164595+0000","last_became_peered":"2024-04-25T17:08:12.164595+0000","last_unstale":"2024-04-25T17:08:13.141635+0000","last_undegraded":"2024-04-25T17:08:13.141635+0000","last_fullsized":"2024-04-25T17:08:13.141635+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:45.020512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495412+0000","last_change":"2024-04-25T17:08:12.153583+0000","last_active":"2024-04-25T17:08:13.495412+0000","last_peered":"2024-04-25T17:08:13.495412+0000","last_clean":"2024-04-25T17:08:13.495412+0000","last_became_active":"2024-04-25T17:08:12.153070+0000","last_became_peered":"2024-04-25T17:08:12.153070+0000","last_unstale":"2024-04-25T17:08:13.495412+0000","last_undegraded":"2024-04-25T17:08:13.495412+0000","last_fullsized":"2024-04-25T17:08:13.495412+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:34.151002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352707+0000","last_change":"2024-04-25T17:08:12.166837+0000","last_active":"2024-04-25T17:08:13.352707+0000","last_peered":"2024-04-25T17:08:13.352707+0000","last_clean":"2024-04-25T17:08:13.352707+0000","last_became_active":"2024-04-25T17:08:12.166477+0000","last_became_peered":"2024-04-25T17:08:12.166477+0000","last_unstale":"2024-04-25T17:08:13.352707+0000","last_undegraded":"2024-04-25T17:08:13.352707+0000","last_fullsized":"2024-04-25T17:08:13.352707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:50.492333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495157+0000","last_change":"2024-04-25T17:08:11.153164+0000","last_active":"2024-04-25T17:08:13.495157+0000","last_peered":"2024-04-25T17:08:13.495157+0000","last_clean":"2024-04-25T17:08:13.495157+0000","last_became_active":"2024-04-25T17:08:11.152189+0000","last_became_peered":"2024-04-25T17:08:11.152189+0000","last_unstale":"2024-04-25T17:08:13.495157+0000","last_undegraded":"2024-04-25T17:08:13.495157+0000","last_fullsized":"2024-04-25T17:08:13.495157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:30.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141002+0000","last_change":"2024-04-25T17:08:12.178117+0000","last_active":"2024-04-25T17:08:13.141002+0000","last_peered":"2024-04-25T17:08:13.141002+0000","last_clean":"2024-04-25T17:08:13.141002+0000","last_became_active":"2024-04-25T17:08:12.177767+0000","last_became_peered":"2024-04-25T17:08:12.177767+0000","last_unstale":"2024-04-25T17:08:13.141002+0000","last_undegraded":"2024-04-25T17:08:13.141002+0000","last_fullsized":"2024-04-25T17:08:13.141002+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:34.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141096+0000","last_change":"2024-04-25T17:08:11.153996+0000","last_active":"2024-04-25T17:08:13.141096+0000","last_peered":"2024-04-25T17:08:13.141096+0000","last_clean":"2024-04-25T17:08:13.141096+0000","last_became_active":"2024-04-25T17:08:11.152993+0000","last_became_peered":"2024-04-25T17:08:11.152993+0000","last_unstale":"2024-04-25T17:08:13.141096+0000","last_undegraded":"2024-04-25T17:08:13.141096+0000","last_fullsized":"2024-04-25T17:08:13.141096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:33.116616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422141+0000","last_change":"2024-04-25T17:08:12.163707+0000","last_active":"2024-04-25T17:08:13.422141+0000","last_peered":"2024-04-25T17:08:13.422141+0000","last_clean":"2024-04-25T17:08:13.422141+0000","last_became_active":"2024-04-25T17:08:12.161868+0000","last_became_peered":"2024-04-25T17:08:12.161868+0000","last_unstale":"2024-04-25T17:08:13.422141+0000","last_undegraded":"2024-04-25T17:08:13.422141+0000","last_fullsized":"2024-04-25T17:08:13.422141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.333036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496262+0000","last_change":"2024-04-25T17:08:11.158101+0000","last_active":"2024-04-25T17:08:13.496262+0000","last_peered":"2024-04-25T17:08:13.496262+0000","last_clean":"2024-04-25T17:08:13.496262+0000","last_became_active":"2024-04-25T17:08:11.157747+0000","last_became_peered":"2024-04-25T17:08:11.157747+0000","last_unstale":"2024-04-25T17:08:13.496262+0000","last_undegraded":"2024-04-25T17:08:13.496262+0000","last_fullsized":"2024-04-25T17:08:13.496262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:03.840823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495644+0000","last_change":"2024-04-25T17:08:12.170278+0000","last_active":"2024-04-25T17:08:13.495644+0000","last_peered":"2024-04-25T17:08:13.495644+0000","last_clean":"2024-04-25T17:08:13.495644+0000","last_became_active":"2024-04-25T17:08:12.169219+0000","last_became_peered":"2024-04-25T17:08:12.169219+0000","last_unstale":"2024-04-25T17:08:13.495644+0000","last_undegraded":"2024-04-25T17:08:13.495644+0000","last_fullsized":"2024-04-25T17:08:13.495644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:35.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496183+0000","last_change":"2024-04-25T17:08:11.153167+0000","last_active":"2024-04-25T17:08:13.496183+0000","last_peered":"2024-04-25T17:08:13.496183+0000","last_clean":"2024-04-25T17:08:13.496183+0000","last_became_active":"2024-04-25T17:08:11.152772+0000","last_became_peered":"2024-04-25T17:08:11.152772+0000","last_unstale":"2024-04-25T17:08:13.496183+0000","last_undegraded":"2024-04-25T17:08:13.496183+0000","last_fullsized":"2024-04-25T17:08:13.496183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:57.198963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495439+0000","last_change":"2024-04-25T17:08:12.163530+0000","last_active":"2024-04-25T17:08:13.495439+0000","last_peered":"2024-04-25T17:08:13.495439+0000","last_clean":"2024-04-25T17:08:13.495439+0000","last_became_active":"2024-04-25T17:08:12.163179+0000","last_became_peered":"2024-04-25T17:08:12.163179+0000","last_unstale":"2024-04-25T17:08:13.495439+0000","last_undegraded":"2024-04-25T17:08:13.495439+0000","last_fullsized":"2024-04-25T17:08:13.495439+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:59.536274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495764+0000","last_change":"2024-04-25T17:08:11.158564+0000","last_active":"2024-04-25T17:08:13.495764+0000","last_peered":"2024-04-25T17:08:13.495764+0000","last_clean":"2024-04-25T17:08:13.495764+0000","last_became_active":"2024-04-25T17:08:11.158143+0000","last_became_peered":"2024-04-25T17:08:11.158143+0000","last_unstale":"2024-04-25T17:08:13.495764+0000","last_undegraded":"2024-04-25T17:08:13.495764+0000","last_fullsized":"2024-04-25T17:08:13.495764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:03.957700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495878+0000","last_change":"2024-04-25T17:08:12.167584+0000","last_active":"2024-04-25T17:08:13.495878+0000","last_peered":"2024-04-25T17:08:13.495878+0000","last_clean":"2024-04-25T17:08:13.495878+0000","last_became_active":"2024-04-25T17:08:12.166717+0000","last_became_peered":"2024-04-25T17:08:12.166717+0000","last_unstale":"2024-04-25T17:08:13.495878+0000","last_undegraded":"2024-04-25T17:08:13.495878+0000","last_fullsized":"2024-04-25T17:08:13.495878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:38.047671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141009+0000","last_change":"2024-04-25T17:08:11.159254+0000","last_active":"2024-04-25T17:08:13.141009+0000","last_peered":"2024-04-25T17:08:13.141009+0000","last_clean":"2024-04-25T17:08:13.141009+0000","last_became_active":"2024-04-25T17:08:11.158864+0000","last_became_peered":"2024-04-25T17:08:11.158864+0000","last_unstale":"2024-04-25T17:08:13.141009+0000","last_undegraded":"2024-04-25T17:08:13.141009+0000","last_fullsized":"2024-04-25T17:08:13.141009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:17.795010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495942+0000","last_change":"2024-04-25T17:08:12.170409+0000","last_active":"2024-04-25T17:08:13.495942+0000","last_peered":"2024-04-25T17:08:13.495942+0000","last_clean":"2024-04-25T17:08:13.495942+0000","last_became_active":"2024-04-25T17:08:12.169510+0000","last_became_peered":"2024-04-25T17:08:12.169510+0000","last_unstale":"2024-04-25T17:08:13.495942+0000","last_undegraded":"2024-04-25T17:08:13.495942+0000","last_fullsized":"2024-04-25T17:08:13.495942+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.310486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495182+0000","last_change":"2024-04-25T17:08:11.150980+0000","last_active":"2024-04-25T17:08:13.495182+0000","last_peered":"2024-04-25T17:08:13.495182+0000","last_clean":"2024-04-25T17:08:13.495182+0000","last_became_active":"2024-04-25T17:08:11.150580+0000","last_became_peered":"2024-04-25T17:08:11.150580+0000","last_unstale":"2024-04-25T17:08:13.495182+0000","last_undegraded":"2024-04-25T17:08:13.495182+0000","last_fullsized":"2024-04-25T17:08:13.495182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:59.539673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140605+0000","last_change":"2024-04-25T17:08:12.171013+0000","last_active":"2024-04-25T17:08:13.140605+0000","last_peered":"2024-04-25T17:08:13.140605+0000","last_clean":"2024-04-25T17:08:13.140605+0000","last_became_active":"2024-04-25T17:08:12.169868+0000","last_became_peered":"2024-04-25T17:08:12.169868+0000","last_unstale":"2024-04-25T17:08:13.140605+0000","last_undegraded":"2024-04-25T17:08:13.140605+0000","last_fullsized":"2024-04-25T17:08:13.140605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:22.418990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495287+0000","last_change":"2024-04-25T17:08:11.157618+0000","last_active":"2024-04-25T17:08:13.495287+0000","last_peered":"2024-04-25T17:08:13.495287+0000","last_clean":"2024-04-25T17:08:13.495287+0000","last_became_active":"2024-04-25T17:08:11.157052+0000","last_became_peered":"2024-04-25T17:08:11.157052+0000","last_unstale":"2024-04-25T17:08:13.495287+0000","last_undegraded":"2024-04-25T17:08:13.495287+0000","last_fullsized":"2024-04-25T17:08:13.495287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:06.032163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494806+0000","last_change":"2024-04-25T17:08:12.151093+0000","last_active":"2024-04-25T17:08:13.494806+0000","last_peered":"2024-04-25T17:08:13.494806+0000","last_clean":"2024-04-25T17:08:13.494806+0000","last_became_active":"2024-04-25T17:08:12.150699+0000","last_became_peered":"2024-04-25T17:08:12.150699+0000","last_unstale":"2024-04-25T17:08:13.494806+0000","last_undegraded":"2024-04-25T17:08:13.494806+0000","last_fullsized":"2024-04-25T17:08:13.494806+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:27.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141389+0000","last_change":"2024-04-25T17:08:11.157673+0000","last_active":"2024-04-25T17:08:13.141389+0000","last_peered":"2024-04-25T17:08:13.141389+0000","last_clean":"2024-04-25T17:08:13.141389+0000","last_became_active":"2024-04-25T17:08:11.157089+0000","last_became_peered":"2024-04-25T17:08:11.157089+0000","last_unstale":"2024-04-25T17:08:13.141389+0000","last_undegraded":"2024-04-25T17:08:13.141389+0000","last_fullsized":"2024-04-25T17:08:13.141389+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:47.586003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495779+0000","last_change":"2024-04-25T17:08:12.170666+0000","last_active":"2024-04-25T17:08:13.495779+0000","last_peered":"2024-04-25T17:08:13.495779+0000","last_clean":"2024-04-25T17:08:13.495779+0000","last_became_active":"2024-04-25T17:08:12.169607+0000","last_became_peered":"2024-04-25T17:08:12.169607+0000","last_unstale":"2024-04-25T17:08:13.495779+0000","last_undegraded":"2024-04-25T17:08:13.495779+0000","last_fullsized":"2024-04-25T17:08:13.495779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121887+0000","last_change":"2024-04-25T17:08:11.153407+0000","last_active":"2024-04-25T17:08:14.121887+0000","last_peered":"2024-04-25T17:08:14.121887+0000","last_clean":"2024-04-25T17:08:14.121887+0000","last_became_active":"2024-04-25T17:08:11.152932+0000","last_became_peered":"2024-04-25T17:08:11.152932+0000","last_unstale":"2024-04-25T17:08:14.121887+0000","last_undegraded":"2024-04-25T17:08:14.121887+0000","last_fullsized":"2024-04-25T17:08:14.121887+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:06.712620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422086+0000","last_change":"2024-04-25T17:08:12.173788+0000","last_active":"2024-04-25T17:08:13.422086+0000","last_peered":"2024-04-25T17:08:13.422086+0000","last_clean":"2024-04-25T17:08:13.422086+0000","last_became_active":"2024-04-25T17:08:12.172714+0000","last_became_peered":"2024-04-25T17:08:12.172714+0000","last_unstale":"2024-04-25T17:08:13.422086+0000","last_undegraded":"2024-04-25T17:08:13.422086+0000","last_fullsized":"2024-04-25T17:08:13.422086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:18.892187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495573+0000","last_change":"2024-04-25T17:08:11.159802+0000","last_active":"2024-04-25T17:08:13.495573+0000","last_peered":"2024-04-25T17:08:13.495573+0000","last_clean":"2024-04-25T17:08:13.495573+0000","last_became_active":"2024-04-25T17:08:11.159431+0000","last_became_peered":"2024-04-25T17:08:11.159431+0000","last_unstale":"2024-04-25T17:08:13.495573+0000","last_undegraded":"2024-04-25T17:08:13.495573+0000","last_fullsized":"2024-04-25T17:08:13.495573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495822+0000","last_change":"2024-04-25T17:08:12.162703+0000","last_active":"2024-04-25T17:08:13.495822+0000","last_peered":"2024-04-25T17:08:13.495822+0000","last_clean":"2024-04-25T17:08:13.495822+0000","last_became_active":"2024-04-25T17:08:12.162361+0000","last_became_peered":"2024-04-25T17:08:12.162361+0000","last_unstale":"2024-04-25T17:08:13.495822+0000","last_undegraded":"2024-04-25T17:08:13.495822+0000","last_fullsized":"2024-04-25T17:08:13.495822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:51.262319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'17","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:18.404153+0000","last_change":"2024-04-25T17:08:11.153938+0000","last_active":"2024-04-25T17:08:18.404153+0000","last_peered":"2024-04-25T17:08:18.404153+0000","last_clean":"2024-04-25T17:08:18.404153+0000","last_became_active":"2024-04-25T17:08:11.153591+0000","last_became_peered":"2024-04-25T17:08:11.153591+0000","last_unstale":"2024-04-25T17:08:18.404153+0000","last_undegraded":"2024-04-25T17:08:18.404153+0000","last_fullsized":"2024-04-25T17:08:18.404153+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":43,"num_write":17,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496204+0000","last_change":"2024-04-25T17:08:12.156045+0000","last_active":"2024-04-25T17:08:13.496204+0000","last_peered":"2024-04-25T17:08:13.496204+0000","last_clean":"2024-04-25T17:08:13.496204+0000","last_became_active":"2024-04-25T17:08:12.155657+0000","last_became_peered":"2024-04-25T17:08:12.155657+0000","last_unstale":"2024-04-25T17:08:13.496204+0000","last_undegraded":"2024-04-25T17:08:13.496204+0000","last_fullsized":"2024-04-25T17:08:13.496204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352496+0000","last_change":"2024-04-25T17:08:11.162009+0000","last_active":"2024-04-25T17:08:13.352496+0000","last_peered":"2024-04-25T17:08:13.352496+0000","last_clean":"2024-04-25T17:08:13.352496+0000","last_became_active":"2024-04-25T17:08:11.161574+0000","last_became_peered":"2024-04-25T17:08:11.161574+0000","last_unstale":"2024-04-25T17:08:13.352496+0000","last_undegraded":"2024-04-25T17:08:13.352496+0000","last_fullsized":"2024-04-25T17:08:13.352496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:16.760355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495191+0000","last_change":"2024-04-25T17:08:12.159522+0000","last_active":"2024-04-25T17:08:13.495191+0000","last_peered":"2024-04-25T17:08:13.495191+0000","last_clean":"2024-04-25T17:08:13.495191+0000","last_became_active":"2024-04-25T17:08:12.159036+0000","last_became_peered":"2024-04-25T17:08:12.159036+0000","last_unstale":"2024-04-25T17:08:13.495191+0000","last_undegraded":"2024-04-25T17:08:13.495191+0000","last_fullsized":"2024-04-25T17:08:13.495191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:32.044488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495796+0000","last_change":"2024-04-25T17:08:11.162487+0000","last_active":"2024-04-25T17:08:13.495796+0000","last_peered":"2024-04-25T17:08:13.495796+0000","last_clean":"2024-04-25T17:08:13.495796+0000","last_became_active":"2024-04-25T17:08:11.161961+0000","last_became_peered":"2024-04-25T17:08:11.161961+0000","last_unstale":"2024-04-25T17:08:13.495796+0000","last_undegraded":"2024-04-25T17:08:13.495796+0000","last_fullsized":"2024-04-25T17:08:13.495796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:02.643683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121943+0000","last_change":"2024-04-25T17:08:12.164107+0000","last_active":"2024-04-25T17:08:14.121943+0000","last_peered":"2024-04-25T17:08:14.121943+0000","last_clean":"2024-04-25T17:08:14.121943+0000","last_became_active":"2024-04-25T17:08:12.163741+0000","last_became_peered":"2024-04-25T17:08:12.163741+0000","last_unstale":"2024-04-25T17:08:14.121943+0000","last_undegraded":"2024-04-25T17:08:14.121943+0000","last_fullsized":"2024-04-25T17:08:14.121943+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.552833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494622+0000","last_change":"2024-04-25T17:08:11.153383+0000","last_active":"2024-04-25T17:08:13.494622+0000","last_peered":"2024-04-25T17:08:13.494622+0000","last_clean":"2024-04-25T17:08:13.494622+0000","last_became_active":"2024-04-25T17:08:11.152573+0000","last_became_peered":"2024-04-25T17:08:11.152573+0000","last_unstale":"2024-04-25T17:08:13.494622+0000","last_undegraded":"2024-04-25T17:08:13.494622+0000","last_fullsized":"2024-04-25T17:08:13.494622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:13.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121535+0000","last_change":"2024-04-25T17:08:12.165939+0000","last_active":"2024-04-25T17:08:14.121535+0000","last_peered":"2024-04-25T17:08:14.121535+0000","last_clean":"2024-04-25T17:08:14.121535+0000","last_became_active":"2024-04-25T17:08:12.165569+0000","last_became_peered":"2024-04-25T17:08:12.165569+0000","last_unstale":"2024-04-25T17:08:14.121535+0000","last_undegraded":"2024-04-25T17:08:14.121535+0000","last_fullsized":"2024-04-25T17:08:14.121535+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:15:16.230457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495142+0000","last_change":"2024-04-25T17:08:11.158927+0000","last_active":"2024-04-25T17:08:13.495142+0000","last_peered":"2024-04-25T17:08:13.495142+0000","last_clean":"2024-04-25T17:08:13.495142+0000","last_became_active":"2024-04-25T17:08:11.158528+0000","last_became_peered":"2024-04-25T17:08:11.158528+0000","last_unstale":"2024-04-25T17:08:13.495142+0000","last_undegraded":"2024-04-25T17:08:13.495142+0000","last_fullsized":"2024-04-25T17:08:13.495142+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.308627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495691+0000","last_change":"2024-04-25T17:08:12.160776+0000","last_active":"2024-04-25T17:08:13.495691+0000","last_peered":"2024-04-25T17:08:13.495691+0000","last_clean":"2024-04-25T17:08:13.495691+0000","last_became_active":"2024-04-25T17:08:12.160409+0000","last_became_peered":"2024-04-25T17:08:12.160409+0000","last_unstale":"2024-04-25T17:08:13.495691+0000","last_undegraded":"2024-04-25T17:08:13.495691+0000","last_fullsized":"2024-04-25T17:08:13.495691+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:51.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495648+0000","last_change":"2024-04-25T17:08:11.153214+0000","last_active":"2024-04-25T17:08:13.495648+0000","last_peered":"2024-04-25T17:08:13.495648+0000","last_clean":"2024-04-25T17:08:13.495648+0000","last_became_active":"2024-04-25T17:08:11.152728+0000","last_became_peered":"2024-04-25T17:08:11.152728+0000","last_unstale":"2024-04-25T17:08:13.495648+0000","last_undegraded":"2024-04-25T17:08:13.495648+0000","last_fullsized":"2024-04-25T17:08:13.495648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.338549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141314+0000","last_change":"2024-04-25T17:08:12.162167+0000","last_active":"2024-04-25T17:08:13.141314+0000","last_peered":"2024-04-25T17:08:13.141314+0000","last_clean":"2024-04-25T17:08:13.141314+0000","last_became_active":"2024-04-25T17:08:12.160896+0000","last_became_peered":"2024-04-25T17:08:12.160896+0000","last_unstale":"2024-04-25T17:08:13.141314+0000","last_undegraded":"2024-04-25T17:08:13.141314+0000","last_fullsized":"2024-04-25T17:08:13.141314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:02.960832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495286+0000","last_change":"2024-04-25T17:08:11.158995+0000","last_active":"2024-04-25T17:08:13.495286+0000","last_peered":"2024-04-25T17:08:13.495286+0000","last_clean":"2024-04-25T17:08:13.495286+0000","last_became_active":"2024-04-25T17:08:11.158644+0000","last_became_peered":"2024-04-25T17:08:11.158644+0000","last_unstale":"2024-04-25T17:08:13.495286+0000","last_undegraded":"2024-04-25T17:08:13.495286+0000","last_fullsized":"2024-04-25T17:08:13.495286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:22.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.149768+0000","last_change":"2024-04-25T17:08:12.161635+0000","last_active":"2024-04-25T17:08:14.149768+0000","last_peered":"2024-04-25T17:08:14.149768+0000","last_clean":"2024-04-25T17:08:14.149768+0000","last_became_active":"2024-04-25T17:08:12.153728+0000","last_became_peered":"2024-04-25T17:08:12.153728+0000","last_unstale":"2024-04-25T17:08:14.149768+0000","last_undegraded":"2024-04-25T17:08:14.149768+0000","last_fullsized":"2024-04-25T17:08:14.149768+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:02.003550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121630+0000","last_change":"2024-04-25T17:08:11.150490+0000","last_active":"2024-04-25T17:08:14.121630+0000","last_peered":"2024-04-25T17:08:14.121630+0000","last_clean":"2024-04-25T17:08:14.121630+0000","last_became_active":"2024-04-25T17:08:11.148391+0000","last_became_peered":"2024-04-25T17:08:11.148391+0000","last_unstale":"2024-04-25T17:08:14.121630+0000","last_undegraded":"2024-04-25T17:08:14.121630+0000","last_fullsized":"2024-04-25T17:08:14.121630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:40.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141295+0000","last_change":"2024-04-25T17:08:12.168136+0000","last_active":"2024-04-25T17:08:13.141295+0000","last_peered":"2024-04-25T17:08:13.141295+0000","last_clean":"2024-04-25T17:08:13.141295+0000","last_became_active":"2024-04-25T17:08:12.167786+0000","last_became_peered":"2024-04-25T17:08:12.167786+0000","last_unstale":"2024-04-25T17:08:13.141295+0000","last_undegraded":"2024-04-25T17:08:13.141295+0000","last_fullsized":"2024-04-25T17:08:13.141295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:22.869503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495978+0000","last_change":"2024-04-25T17:08:12.159952+0000","last_active":"2024-04-25T17:08:13.495978+0000","last_peered":"2024-04-25T17:08:13.495978+0000","last_clean":"2024-04-25T17:08:13.495978+0000","last_became_active":"2024-04-25T17:08:12.159429+0000","last_became_peered":"2024-04-25T17:08:12.159429+0000","last_unstale":"2024-04-25T17:08:13.495978+0000","last_undegraded":"2024-04-25T17:08:13.495978+0000","last_fullsized":"2024-04-25T17:08:13.495978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:13:30.521084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495045+0000","last_change":"2024-04-25T17:08:11.149085+0000","last_active":"2024-04-25T17:08:13.495045+0000","last_peered":"2024-04-25T17:08:13.495045+0000","last_clean":"2024-04-25T17:08:13.495045+0000","last_became_active":"2024-04-25T17:08:11.148712+0000","last_became_peered":"2024-04-25T17:08:11.148712+0000","last_unstale":"2024-04-25T17:08:13.495045+0000","last_undegraded":"2024-04-25T17:08:13.495045+0000","last_fullsized":"2024-04-25T17:08:13.495045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:12.764034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140954+0000","last_change":"2024-04-25T17:08:12.178154+0000","last_active":"2024-04-25T17:08:13.140954+0000","last_peered":"2024-04-25T17:08:13.140954+0000","last_clean":"2024-04-25T17:08:13.140954+0000","last_became_active":"2024-04-25T17:08:12.177773+0000","last_became_peered":"2024-04-25T17:08:12.177773+0000","last_unstale":"2024-04-25T17:08:13.140954+0000","last_undegraded":"2024-04-25T17:08:13.140954+0000","last_fullsized":"2024-04-25T17:08:13.140954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.962410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494975+0000","last_change":"2024-04-25T17:08:11.152932+0000","last_active":"2024-04-25T17:08:13.494975+0000","last_peered":"2024-04-25T17:08:13.494975+0000","last_clean":"2024-04-25T17:08:13.494975+0000","last_became_active":"2024-04-25T17:08:11.152530+0000","last_became_peered":"2024-04-25T17:08:11.152530+0000","last_unstale":"2024-04-25T17:08:13.494975+0000","last_undegraded":"2024-04-25T17:08:13.494975+0000","last_fullsized":"2024-04-25T17:08:13.494975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121425+0000","last_change":"2024-04-25T17:06:55.951326+0000","last_active":"2024-04-25T17:08:14.121425+0000","last_peered":"2024-04-25T17:08:14.121425+0000","last_clean":"2024-04-25T17:08:14.121425+0000","last_became_active":"2024-04-25T17:06:55.946641+0000","last_became_peered":"2024-04-25T17:06:55.946641+0000","last_unstale":"2024-04-25T17:08:14.121425+0000","last_undegraded":"2024-04-25T17:08:14.121425+0000","last_fullsized":"2024-04-25T17:08:14.121425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_clean_scrub_stamp":"2024-04-25T17:03:27.394276+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:58.308843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494922+0000","last_change":"2024-04-25T17:08:12.162065+0000","last_active":"2024-04-25T17:08:13.494922+0000","last_peered":"2024-04-25T17:08:13.494922+0000","last_clean":"2024-04-25T17:08:13.494922+0000","last_became_active":"2024-04-25T17:08:12.161733+0000","last_became_peered":"2024-04-25T17:08:12.161733+0000","last_unstale":"2024-04-25T17:08:13.494922+0000","last_undegraded":"2024-04-25T17:08:13.494922+0000","last_fullsized":"2024-04-25T17:08:13.494922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:28.870514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121612+0000","last_change":"2024-04-25T17:08:11.155941+0000","last_active":"2024-04-25T17:08:14.121612+0000","last_peered":"2024-04-25T17:08:14.121612+0000","last_clean":"2024-04-25T17:08:14.121612+0000","last_became_active":"2024-04-25T17:08:11.155381+0000","last_became_peered":"2024-04-25T17:08:11.155381+0000","last_unstale":"2024-04-25T17:08:14.121612+0000","last_undegraded":"2024-04-25T17:08:14.121612+0000","last_fullsized":"2024-04-25T17:08:14.121612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:28.423462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496103+0000","last_change":"2024-04-25T17:08:11.158960+0000","last_active":"2024-04-25T17:08:13.496103+0000","last_peered":"2024-04-25T17:08:13.496103+0000","last_clean":"2024-04-25T17:08:13.496103+0000","last_became_active":"2024-04-25T17:08:11.158609+0000","last_became_peered":"2024-04-25T17:08:11.158609+0000","last_unstale":"2024-04-25T17:08:13.496103+0000","last_undegraded":"2024-04-25T17:08:13.496103+0000","last_fullsized":"2024-04-25T17:08:13.496103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:19.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495147+0000","last_change":"2024-04-25T17:08:12.161483+0000","last_active":"2024-04-25T17:08:13.495147+0000","last_peered":"2024-04-25T17:08:13.495147+0000","last_clean":"2024-04-25T17:08:13.495147+0000","last_became_active":"2024-04-25T17:08:12.158586+0000","last_became_peered":"2024-04-25T17:08:12.158586+0000","last_unstale":"2024-04-25T17:08:13.495147+0000","last_undegraded":"2024-04-25T17:08:13.495147+0000","last_fullsized":"2024-04-25T17:08:13.495147+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.551328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'3","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:16.146220+0000","last_change":"2024-04-25T17:08:11.161497+0000","last_active":"2024-04-25T17:08:16.146220+0000","last_peered":"2024-04-25T17:08:16.146220+0000","last_clean":"2024-04-25T17:08:16.146220+0000","last_became_active":"2024-04-25T17:08:11.160990+0000","last_became_peered":"2024-04-25T17:08:11.160990+0000","last_unstale":"2024-04-25T17:08:16.146220+0000","last_undegraded":"2024-04-25T17:08:16.146220+0000","last_fullsized":"2024-04-25T17:08:16.146220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:59.682133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141148+0000","last_change":"2024-04-25T17:08:12.171214+0000","last_active":"2024-04-25T17:08:13.141148+0000","last_peered":"2024-04-25T17:08:13.141148+0000","last_clean":"2024-04-25T17:08:13.141148+0000","last_became_active":"2024-04-25T17:08:12.170877+0000","last_became_peered":"2024-04-25T17:08:12.170877+0000","last_unstale":"2024-04-25T17:08:13.141148+0000","last_undegraded":"2024-04-25T17:08:13.141148+0000","last_fullsized":"2024-04-25T17:08:13.141148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:18.442691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.494978+0000","last_change":"2024-04-25T17:08:11.154819+0000","last_active":"2024-04-25T17:08:13.494978+0000","last_peered":"2024-04-25T17:08:13.494978+0000","last_clean":"2024-04-25T17:08:13.494978+0000","last_became_active":"2024-04-25T17:08:11.154006+0000","last_became_peered":"2024-04-25T17:08:11.154006+0000","last_unstale":"2024-04-25T17:08:13.494978+0000","last_undegraded":"2024-04-25T17:08:13.494978+0000","last_fullsized":"2024-04-25T17:08:13.494978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.063058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.422056+0000","last_change":"2024-04-25T17:08:12.167301+0000","last_active":"2024-04-25T17:08:13.422056+0000","last_peered":"2024-04-25T17:08:13.422056+0000","last_clean":"2024-04-25T17:08:13.422056+0000","last_became_active":"2024-04-25T17:08:12.166944+0000","last_became_peered":"2024-04-25T17:08:12.166944+0000","last_unstale":"2024-04-25T17:08:13.422056+0000","last_undegraded":"2024-04-25T17:08:13.422056+0000","last_fullsized":"2024-04-25T17:08:13.422056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:46.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495108+0000","last_change":"2024-04-25T17:08:11.156616+0000","last_active":"2024-04-25T17:08:13.495108+0000","last_peered":"2024-04-25T17:08:13.495108+0000","last_clean":"2024-04-25T17:08:13.495108+0000","last_became_active":"2024-04-25T17:08:11.156256+0000","last_became_peered":"2024-04-25T17:08:11.156256+0000","last_unstale":"2024-04-25T17:08:13.495108+0000","last_undegraded":"2024-04-25T17:08:13.495108+0000","last_fullsized":"2024-04-25T17:08:13.495108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:37.539773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495319+0000","last_change":"2024-04-25T17:08:12.175216+0000","last_active":"2024-04-25T17:08:13.495319+0000","last_peered":"2024-04-25T17:08:13.495319+0000","last_clean":"2024-04-25T17:08:13.495319+0000","last_became_active":"2024-04-25T17:08:12.174521+0000","last_became_peered":"2024-04-25T17:08:12.174521+0000","last_unstale":"2024-04-25T17:08:13.495319+0000","last_undegraded":"2024-04-25T17:08:13.495319+0000","last_fullsized":"2024-04-25T17:08:13.495319+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.322733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140606+0000","last_change":"2024-04-25T17:08:11.162287+0000","last_active":"2024-04-25T17:08:13.140606+0000","last_peered":"2024-04-25T17:08:13.140606+0000","last_clean":"2024-04-25T17:08:13.140606+0000","last_became_active":"2024-04-25T17:08:11.158297+0000","last_became_peered":"2024-04-25T17:08:11.158297+0000","last_unstale":"2024-04-25T17:08:13.140606+0000","last_undegraded":"2024-04-25T17:08:13.140606+0000","last_fullsized":"2024-04-25T17:08:13.140606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:15.758880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141260+0000","last_change":"2024-04-25T17:08:12.163535+0000","last_active":"2024-04-25T17:08:13.141260+0000","last_peered":"2024-04-25T17:08:13.141260+0000","last_clean":"2024-04-25T17:08:13.141260+0000","last_became_active":"2024-04-25T17:08:12.162848+0000","last_became_peered":"2024-04-25T17:08:12.162848+0000","last_unstale":"2024-04-25T17:08:13.141260+0000","last_undegraded":"2024-04-25T17:08:13.141260+0000","last_fullsized":"2024-04-25T17:08:13.141260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:50.288869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352187+0000","last_change":"2024-04-25T17:08:11.162780+0000","last_active":"2024-04-25T17:08:13.352187+0000","last_peered":"2024-04-25T17:08:13.352187+0000","last_clean":"2024-04-25T17:08:13.352187+0000","last_became_active":"2024-04-25T17:08:11.162385+0000","last_became_peered":"2024-04-25T17:08:11.162385+0000","last_unstale":"2024-04-25T17:08:13.352187+0000","last_undegraded":"2024-04-25T17:08:13.352187+0000","last_fullsized":"2024-04-25T17:08:13.352187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:25.420740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140869+0000","last_change":"2024-04-25T17:08:12.170828+0000","last_active":"2024-04-25T17:08:13.140869+0000","last_peered":"2024-04-25T17:08:13.140869+0000","last_clean":"2024-04-25T17:08:13.140869+0000","last_became_active":"2024-04-25T17:08:12.170479+0000","last_became_peered":"2024-04-25T17:08:12.170479+0000","last_unstale":"2024-04-25T17:08:13.140869+0000","last_undegraded":"2024-04-25T17:08:13.140869+0000","last_fullsized":"2024-04-25T17:08:13.140869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.947118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495662+0000","last_change":"2024-04-25T17:08:11.156754+0000","last_active":"2024-04-25T17:08:13.495662+0000","last_peered":"2024-04-25T17:08:13.495662+0000","last_clean":"2024-04-25T17:08:13.495662+0000","last_became_active":"2024-04-25T17:08:11.156414+0000","last_became_peered":"2024-04-25T17:08:11.156414+0000","last_unstale":"2024-04-25T17:08:13.495662+0000","last_undegraded":"2024-04-25T17:08:13.495662+0000","last_fullsized":"2024-04-25T17:08:13.495662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:10.364234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.121978+0000","last_change":"2024-04-25T17:08:12.162032+0000","last_active":"2024-04-25T17:08:14.121978+0000","last_peered":"2024-04-25T17:08:14.121978+0000","last_clean":"2024-04-25T17:08:14.121978+0000","last_became_active":"2024-04-25T17:08:12.161017+0000","last_became_peered":"2024-04-25T17:08:12.161017+0000","last_unstale":"2024-04-25T17:08:14.121978+0000","last_undegraded":"2024-04-25T17:08:14.121978+0000","last_fullsized":"2024-04-25T17:08:14.121978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.472984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352660+0000","last_change":"2024-04-25T17:08:11.149550+0000","last_active":"2024-04-25T17:08:13.352660+0000","last_peered":"2024-04-25T17:08:13.352660+0000","last_clean":"2024-04-25T17:08:13.352660+0000","last_became_active":"2024-04-25T17:08:11.149082+0000","last_became_peered":"2024-04-25T17:08:11.149082+0000","last_unstale":"2024-04-25T17:08:13.352660+0000","last_undegraded":"2024-04-25T17:08:13.352660+0000","last_fullsized":"2024-04-25T17:08:13.352660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:14.565877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122122+0000","last_change":"2024-04-25T17:08:12.160331+0000","last_active":"2024-04-25T17:08:14.122122+0000","last_peered":"2024-04-25T17:08:14.122122+0000","last_clean":"2024-04-25T17:08:14.122122+0000","last_became_active":"2024-04-25T17:08:12.159931+0000","last_became_peered":"2024-04-25T17:08:12.159931+0000","last_unstale":"2024-04-25T17:08:14.122122+0000","last_undegraded":"2024-04-25T17:08:14.122122+0000","last_fullsized":"2024-04-25T17:08:14.122122+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:37.246974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.496227+0000","last_change":"2024-04-25T17:08:11.163118+0000","last_active":"2024-04-25T17:08:13.496227+0000","last_peered":"2024-04-25T17:08:13.496227+0000","last_clean":"2024-04-25T17:08:13.496227+0000","last_became_active":"2024-04-25T17:08:11.161691+0000","last_became_peered":"2024-04-25T17:08:11.161691+0000","last_unstale":"2024-04-25T17:08:13.496227+0000","last_undegraded":"2024-04-25T17:08:13.496227+0000","last_fullsized":"2024-04-25T17:08:13.496227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:32.006098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:15.168333+0000","last_change":"2024-04-25T17:08:12.168387+0000","last_active":"2024-04-25T17:08:15.168333+0000","last_peered":"2024-04-25T17:08:15.168333+0000","last_clean":"2024-04-25T17:08:15.168333+0000","last_became_active":"2024-04-25T17:08:12.168008+0000","last_became_peered":"2024-04-25T17:08:12.168008+0000","last_unstale":"2024-04-25T17:08:15.168333+0000","last_undegraded":"2024-04-25T17:08:15.168333+0000","last_fullsized":"2024-04-25T17:08:15.168333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:08.438945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495492+0000","last_change":"2024-04-25T17:08:11.149181+0000","last_active":"2024-04-25T17:08:13.495492+0000","last_peered":"2024-04-25T17:08:13.495492+0000","last_clean":"2024-04-25T17:08:13.495492+0000","last_became_active":"2024-04-25T17:08:11.148847+0000","last_became_peered":"2024-04-25T17:08:11.148847+0000","last_unstale":"2024-04-25T17:08:13.495492+0000","last_undegraded":"2024-04-25T17:08:13.495492+0000","last_fullsized":"2024-04-25T17:08:13.495492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:40.720578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:14.122115+0000","last_change":"2024-04-25T17:08:12.162173+0000","last_active":"2024-04-25T17:08:14.122115+0000","last_peered":"2024-04-25T17:08:14.122115+0000","last_clean":"2024-04-25T17:08:14.122115+0000","last_became_active":"2024-04-25T17:08:12.161818+0000","last_became_peered":"2024-04-25T17:08:12.161818+0000","last_unstale":"2024-04-25T17:08:14.122115+0000","last_undegraded":"2024-04-25T17:08:14.122115+0000","last_fullsized":"2024-04-25T17:08:14.122115+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:28.872937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141351+0000","last_change":"2024-04-25T17:08:12.165542+0000","last_active":"2024-04-25T17:08:13.141351+0000","last_peered":"2024-04-25T17:08:13.141351+0000","last_clean":"2024-04-25T17:08:13.141351+0000","last_became_active":"2024-04-25T17:08:12.165190+0000","last_became_peered":"2024-04-25T17:08:12.165190+0000","last_unstale":"2024-04-25T17:08:13.141351+0000","last_undegraded":"2024-04-25T17:08:13.141351+0000","last_fullsized":"2024-04-25T17:08:13.141351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:51.650953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352355+0000","last_change":"2024-04-25T17:08:12.161429+0000","last_active":"2024-04-25T17:08:13.352355+0000","last_peered":"2024-04-25T17:08:13.352355+0000","last_clean":"2024-04-25T17:08:13.352355+0000","last_became_active":"2024-04-25T17:08:12.159184+0000","last_became_peered":"2024-04-25T17:08:12.159184+0000","last_unstale":"2024-04-25T17:08:13.352355+0000","last_undegraded":"2024-04-25T17:08:13.352355+0000","last_fullsized":"2024-04-25T17:08:13.352355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:56:21.366775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352298+0000","last_change":"2024-04-25T17:08:12.169948+0000","last_active":"2024-04-25T17:08:13.352298+0000","last_peered":"2024-04-25T17:08:13.352298+0000","last_clean":"2024-04-25T17:08:13.352298+0000","last_became_active":"2024-04-25T17:08:12.169592+0000","last_became_peered":"2024-04-25T17:08:12.169592+0000","last_unstale":"2024-04-25T17:08:13.352298+0000","last_undegraded":"2024-04-25T17:08:13.352298+0000","last_fullsized":"2024-04-25T17:08:13.352298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:22.749954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.421852+0000","last_change":"2024-04-25T17:08:12.169785+0000","last_active":"2024-04-25T17:08:13.421852+0000","last_peered":"2024-04-25T17:08:13.421852+0000","last_clean":"2024-04-25T17:08:13.421852+0000","last_became_active":"2024-04-25T17:08:12.169226+0000","last_became_peered":"2024-04-25T17:08:12.169226+0000","last_unstale":"2024-04-25T17:08:13.421852+0000","last_undegraded":"2024-04-25T17:08:13.421852+0000","last_fullsized":"2024-04-25T17:08:13.421852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.232112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495028+0000","last_change":"2024-04-25T17:08:12.157992+0000","last_active":"2024-04-25T17:08:13.495028+0000","last_peered":"2024-04-25T17:08:13.495028+0000","last_clean":"2024-04-25T17:08:13.495028+0000","last_became_active":"2024-04-25T17:08:12.157636+0000","last_became_peered":"2024-04-25T17:08:12.157636+0000","last_unstale":"2024-04-25T17:08:13.495028+0000","last_undegraded":"2024-04-25T17:08:13.495028+0000","last_fullsized":"2024-04-25T17:08:13.495028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:50.530074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141353+0000","last_change":"2024-04-25T17:08:12.160977+0000","last_active":"2024-04-25T17:08:13.141353+0000","last_peered":"2024-04-25T17:08:13.141353+0000","last_clean":"2024-04-25T17:08:13.141353+0000","last_became_active":"2024-04-25T17:08:12.160571+0000","last_became_peered":"2024-04-25T17:08:12.160571+0000","last_unstale":"2024-04-25T17:08:13.141353+0000","last_undegraded":"2024-04-25T17:08:13.141353+0000","last_fullsized":"2024-04-25T17:08:13.141353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.882628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.352838+0000","last_change":"2024-04-25T17:08:12.158566+0000","last_active":"2024-04-25T17:08:13.352838+0000","last_peered":"2024-04-25T17:08:13.352838+0000","last_clean":"2024-04-25T17:08:13.352838+0000","last_became_active":"2024-04-25T17:08:12.157911+0000","last_became_peered":"2024-04-25T17:08:12.157911+0000","last_unstale":"2024-04-25T17:08:13.352838+0000","last_undegraded":"2024-04-25T17:08:13.352838+0000","last_fullsized":"2024-04-25T17:08:13.352838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:49.665779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141146+0000","last_change":"2024-04-25T17:08:12.174931+0000","last_active":"2024-04-25T17:08:13.141146+0000","last_peered":"2024-04-25T17:08:13.141146+0000","last_clean":"2024-04-25T17:08:13.141146+0000","last_became_active":"2024-04-25T17:08:12.174571+0000","last_became_peered":"2024-04-25T17:08:12.174571+0000","last_unstale":"2024-04-25T17:08:13.141146+0000","last_undegraded":"2024-04-25T17:08:13.141146+0000","last_fullsized":"2024-04-25T17:08:13.141146+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:55.832578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495557+0000","last_change":"2024-04-25T17:08:12.165592+0000","last_active":"2024-04-25T17:08:13.495557+0000","last_peered":"2024-04-25T17:08:13.495557+0000","last_clean":"2024-04-25T17:08:13.495557+0000","last_became_active":"2024-04-25T17:08:12.165273+0000","last_became_peered":"2024-04-25T17:08:12.165273+0000","last_unstale":"2024-04-25T17:08:13.495557+0000","last_undegraded":"2024-04-25T17:08:13.495557+0000","last_fullsized":"2024-04-25T17:08:13.495557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:25.199657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.140590+0000","last_change":"2024-04-25T17:08:12.174095+0000","last_active":"2024-04-25T17:08:13.140590+0000","last_peered":"2024-04-25T17:08:13.140590+0000","last_clean":"2024-04-25T17:08:13.140590+0000","last_became_active":"2024-04-25T17:08:12.172786+0000","last_became_peered":"2024-04-25T17:08:12.172786+0000","last_unstale":"2024-04-25T17:08:13.140590+0000","last_undegraded":"2024-04-25T17:08:13.140590+0000","last_fullsized":"2024-04-25T17:08:13.140590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.736300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495785+0000","last_change":"2024-04-25T17:08:12.158772+0000","last_active":"2024-04-25T17:08:13.495785+0000","last_peered":"2024-04-25T17:08:13.495785+0000","last_clean":"2024-04-25T17:08:13.495785+0000","last_became_active":"2024-04-25T17:08:12.158417+0000","last_became_peered":"2024-04-25T17:08:12.158417+0000","last_unstale":"2024-04-25T17:08:13.495785+0000","last_undegraded":"2024-04-25T17:08:13.495785+0000","last_fullsized":"2024-04-25T17:08:13.495785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:21.837887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.495613+0000","last_change":"2024-04-25T17:08:12.162384+0000","last_active":"2024-04-25T17:08:13.495613+0000","last_peered":"2024-04-25T17:08:13.495613+0000","last_clean":"2024-04-25T17:08:13.495613+0000","last_became_active":"2024-04-25T17:08:12.161997+0000","last_became_peered":"2024-04-25T17:08:12.161997+0000","last_unstale":"2024-04-25T17:08:13.495613+0000","last_undegraded":"2024-04-25T17:08:13.495613+0000","last_fullsized":"2024-04-25T17:08:13.495613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:17.358171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T17:08:13.141217+0000","last_change":"2024-04-25T17:08:12.159015+0000","last_active":"2024-04-25T17:08:13.141217+0000","last_peered":"2024-04-25T17:08:13.141217+0000","last_clean":"2024-04-25T17:08:13.141217+0000","last_became_active":"2024-04-25T17:08:12.157999+0000","last_became_peered":"2024-04-25T17:08:12.157999+0000","last_unstale":"2024-04-25T17:08:13.141217+0000","last_undegraded":"2024-04-25T17:08:13.141217+0000","last_fullsized":"2024-04-25T17:08:13.141217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:00.291221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280965,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":38991,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":20,"num_read_kb":49,"num_write":60,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":116973,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678029,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":389120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":1017992,"data_compressed":37573,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037785,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":815104,"data_stored":389702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234014,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":819200,"data_stored":389724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430244,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":868352,"data_stored":980187,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":815104,"data_stored":389237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":856064,"data_stored":427313,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":815104,"data_stored":389210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":410049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":823296,"data_stored":389341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":815104,"data_stored":389142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":979670,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":38193,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38227,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":38387,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T17:08:22.986 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T17:08:22.986 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T17:08:22.986 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T17:08:23.260 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:24.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:24 smithi096 ceph-mon[29131]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:24.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:24 smithi096 ceph-mon[29131]: pgmap v286: 97 pgs: 97 active+clean; 615 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-04-25T17:08:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:24 smithi097 ceph-mon[32602]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:24 smithi097 ceph-mon[32602]: pgmap v286: 97 pgs: 97 active+clean; 615 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-04-25T17:08:24.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:24 smithi171 ceph-mon[33189]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:08:24.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:24 smithi171 ceph-mon[33189]: pgmap v286: 97 pgs: 97 active+clean; 615 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-04-25T17:08:25.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 -- 172.21.15.96:0/1561084900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec109fc0 msgr2=0x7f73ec10c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:25.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 --2- 172.21.15.96:0/1561084900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec109fc0 0x7f73ec10c3e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f73e400b0a0 tx=0x7f73e400b3b0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 -- 172.21.15.96:0/1561084900 shutdown_connections 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 --2- 172.21.15.96:0/1561084900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec109fc0 0x7f73ec10c3e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 --2- 172.21.15.96:0/1561084900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73ec104d00 0x7f73ec107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 --2- 172.21.15.96:0/1561084900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec107660 0x7f73ec109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 -- 172.21.15.96:0/1561084900 >> 172.21.15.96:0/1561084900 conn(0x7f73ec0fe740 msgr2=0x7f73ec100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 -- 172.21.15.96:0/1561084900 shutdown_connections 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.092+0000 7f73f2748700 1 -- 172.21.15.96:0/1561084900 wait complete. 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 Processor -- start 2024-04-25T17:08:25.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 -- start start 2024-04-25T17:08:25.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec104d00 0x7f73ec19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:25.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73ec107660 0x7f73ec19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:25.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec109fc0 0x7f73ec1a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:25.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10fcd0 con 0x7f73ec104d00 2024-04-25T17:08:25.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10fb50 con 0x7f73ec109fc0 2024-04-25T17:08:25.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73f2748700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10fe50 con 0x7f73ec107660 2024-04-25T17:08:25.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec104d00 0x7f73ec19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec104d00 0x7f73ec19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47732/0 (socket says 172.21.15.96:47732) 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.093+0000 7f73ebfff700 1 -- 172.21.15.96:0/1130956552 learned_addr learned my addr 172.21.15.96:0/1130956552 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73eb7fe700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73ec107660 0x7f73ec19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73f0ce5700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec109fc0 0x7f73ec1a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73ebfff700 1 -- 172.21.15.96:0/1130956552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73ec107660 msgr2=0x7f73ec19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73ebfff700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73ec107660 0x7f73ec19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73ebfff700 1 -- 172.21.15.96:0/1130956552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec109fc0 msgr2=0x7f73ec1a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73ebfff700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec109fc0 0x7f73ec1a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73ebfff700 1 -- 172.21.15.96:0/1130956552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec1a22e0 con 0x7f73ec104d00 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73ebfff700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec104d00 0x7f73ec19a780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00b500 tx=0x7f73dc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73e97fa700 1 -- 172.21.15.96:0/1130956552 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc015020 con 0x7f73ec104d00 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73e97fa700 1 -- 172.21.15.96:0/1130956552 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73dc011ce0 con 0x7f73ec104d00 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec1a25d0 con 0x7f73ec104d00 2024-04-25T17:08:25.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73e97fa700 1 -- 172.21.15.96:0/1130956552 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc010d40 con 0x7f73ec104d00 2024-04-25T17:08:25.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.094+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a2be0 con 0x7f73ec104d00 2024-04-25T17:08:25.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.096+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec051190 con 0x7f73ec104d00 2024-04-25T17:08:25.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.097+0000 7f73e97fa700 1 -- 172.21.15.96:0/1130956552 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f73dc018020 con 0x7f73ec104d00 2024-04-25T17:08:25.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.097+0000 7f73e97fa700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f73d40afc90 0x7f73d40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:25.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.097+0000 7f73eb7fe700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f73d40afc90 0x7f73d40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:25.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.098+0000 7f73e97fa700 1 -- 172.21.15.96:0/1130956552 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f73dc0de190 con 0x7f73ec104d00 2024-04-25T17:08:25.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.098+0000 7f73eb7fe700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f73d40afc90 0x7f73d40b21a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005950 tx=0x7f73e000a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:25.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.102+0000 7f73e97fa700 1 -- 172.21.15.96:0/1130956552 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73dc01f020 con 0x7f73ec104d00 2024-04-25T17:08:25.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.381+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f73ec02e510 con 0x7f73ec104d00 2024-04-25T17:08:25.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.384+0000 7f73e97fa700 1 -- 172.21.15.96:0/1130956552 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f73dc0a6ed0 con 0x7f73ec104d00 2024-04-25T17:08:25.384 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-04-25T17:08:25.384 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAJjipm8G3YFhAAdnDxedqWw0vBxeZiOJjgQw== 2024-04-25T17:08:25.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f73d40afc90 msgr2=0x7f73d40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:25.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f73d40afc90 0x7f73d40b21a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005950 tx=0x7f73e000a300 comp rx=0 tx=0).stop 2024-04-25T17:08:25.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec104d00 msgr2=0x7f73ec19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:25.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec104d00 0x7f73ec19a780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00b500 tx=0x7f73dc00b810 comp rx=0 tx=0).stop 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 shutdown_connections 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73ec109fc0 0x7f73ec1a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f73d40afc90 0x7f73d40b21a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73ec104d00 0x7f73ec19a780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 --2- 172.21.15.96:0/1130956552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73ec107660 0x7f73ec19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 >> 172.21.15.96:0/1130956552 conn(0x7f73ec0fe740 msgr2=0x7f73ec1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 shutdown_connections 2024-04-25T17:08:25.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:25.386+0000 7f73f2748700 1 -- 172.21.15.96:0/1130956552 wait complete. 2024-04-25T17:08:26.187 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:08:26.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T17:08:26.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T17:08:26.223 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T17:08:26.223 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T17:08:26.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T17:08:26.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:26 smithi096 ceph-mon[29131]: pgmap v287: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T17:08:26.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:26 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1130956552' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T17:08:26.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:26 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1130956552' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T17:08:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:26 smithi097 ceph-mon[32602]: pgmap v287: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T17:08:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:26 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1130956552' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T17:08:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:26 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1130956552' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T17:08:26.499 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:26 smithi171 ceph-mon[33189]: pgmap v287: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T17:08:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:26 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1130956552' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T17:08:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:26 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1130956552' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T17:08:28.257 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:28 smithi096 ceph-mon[29131]: pgmap v288: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-04-25T17:08:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 -- 172.21.15.96:0/2701797558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 msgr2=0x7f0be8103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 --2- 172.21.15.96:0/2701797558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be8103f80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0099c0 tx=0x7f0bdc009cd0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 -- 172.21.15.96:0/2701797558 shutdown_connections 2024-04-25T17:08:28.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 --2- 172.21.15.96:0/2701797558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8108630 0x7f0be8110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 --2- 172.21.15.96:0/2701797558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be8103f80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 --2- 172.21.15.96:0/2701797558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be8107ce0 0x7f0be81080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 -- 172.21.15.96:0/2701797558 >> 172.21.15.96:0/2701797558 conn(0x7f0be80fe5c0 msgr2=0x7f0be81009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 -- 172.21.15.96:0/2701797558 shutdown_connections 2024-04-25T17:08:28.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.265+0000 7f0bedf49700 1 -- 172.21.15.96:0/2701797558 wait complete. 2024-04-25T17:08:28.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.266+0000 7f0bedf49700 1 Processor -- start 2024-04-25T17:08:28.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.266+0000 7f0bedf49700 1 -- start start 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0bedf49700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0bedf49700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be8107ce0 0x7f0be819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0bedf49700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8108630 0x7f0be819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0bedf49700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8113250 con 0x7f0be8103b70 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0bedf49700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81130d0 con 0x7f0be8108630 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0bedf49700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0be81133d0 con 0x7f0be8107ce0 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53354/0 (socket says 172.21.15.96:53354) 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 -- 172.21.15.96:0/1174074823 learned_addr learned my addr 172.21.15.96:0/1174074823 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be7fff700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8108630 0x7f0be819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be6ffd700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be8107ce0 0x7f0be819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 -- 172.21.15.96:0/1174074823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be8107ce0 msgr2=0x7f0be819c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be8107ce0 0x7f0be819c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 -- 172.21.15.96:0/1174074823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8108630 msgr2=0x7f0be819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8108630 0x7f0be819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 -- 172.21.15.96:0/1174074823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be81a6e30 con 0x7f0be8103b70 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be6ffd700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be8107ce0 0x7f0be819c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.267+0000 7f0be77fe700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be819bb50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00b580 tx=0x7f0bdc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.268+0000 7f0be4ff9700 1 -- 172.21.15.96:0/1174074823 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc027070 con 0x7f0be8103b70 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.268+0000 7f0be4ff9700 1 -- 172.21.15.96:0/1174074823 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bdc003f20 con 0x7f0be8103b70 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.268+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be81a70c0 con 0x7f0be8103b70 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.268+0000 7f0be4ff9700 1 -- 172.21.15.96:0/1174074823 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc022d70 con 0x7f0be8103b70 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.268+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be81a7510 con 0x7f0be8103b70 2024-04-25T17:08:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.269+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be8051190 con 0x7f0be8103b70 2024-04-25T17:08:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.270+0000 7f0be4ff9700 1 -- 172.21.15.96:0/1174074823 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0bdc022500 con 0x7f0be8103b70 2024-04-25T17:08:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.271+0000 7f0be4ff9700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0bd00afc10 0x7f0bd00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.271+0000 7f0be6ffd700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0bd00afc10 0x7f0bd00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.271+0000 7f0be4ff9700 1 -- 172.21.15.96:0/1174074823 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f0bdc0dfc90 con 0x7f0be8103b70 2024-04-25T17:08:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.272+0000 7f0be6ffd700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0bd00afc10 0x7f0bd00b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005950 tx=0x7f0bd80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:28.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.275+0000 7f0be4ff9700 1 -- 172.21.15.96:0/1174074823 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bdc013070 con 0x7f0be8103b70 2024-04-25T17:08:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:28 smithi097 ceph-mon[32602]: pgmap v288: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-04-25T17:08:28.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:28 smithi171 ceph-mon[33189]: pgmap v288: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-04-25T17:08:28.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.541+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0be802e510 con 0x7f0be8103b70 2024-04-25T17:08:28.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.548+0000 7f0be4ff9700 1 -- 172.21.15.96:0/1174074823 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284478 (secure 0 0 0) 0x7f0bdc0a88d0 con 0x7f0be8103b70 2024-04-25T17:08:28.552 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:28.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0bd00afc10 msgr2=0x7f0bd00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:28.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0bd00afc10 0x7f0bd00b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8005950 tx=0x7f0bd80058e0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 msgr2=0x7f0be819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:28.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be819bb50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00b580 tx=0x7f0bdc0047b0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 shutdown_connections 2024-04-25T17:08:28.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8108630 0x7f0be819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0bd00afc10 0x7f0bd00b2120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be8103b70 0x7f0be819bb50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 --2- 172.21.15.96:0/1174074823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0be8107ce0 0x7f0be819c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:28.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 >> 172.21.15.96:0/1174074823 conn(0x7f0be80fe5c0 msgr2=0x7f0be8109590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:28.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 shutdown_connections 2024-04-25T17:08:28.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:28.551+0000 7f0bedf49700 1 -- 172.21.15.96:0/1174074823 wait complete. 2024-04-25T17:08:29.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:29 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1174074823' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T17:08:29.356 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3530143863},{"type":"v1","addr":"172.21.15.96:6801","nonce":3530143863}]},"active_addr":"172.21.15.96:6801/3530143863","active_change":"2024-04-25T17:01:02.956259+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.96:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":328848234}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3419689135}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":1944193211}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":559998201}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":1683660320}]}]} 2024-04-25T17:08:29.361 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T17:08:29.361 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T17:08:29.361 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T17:08:29.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:29 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1174074823' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T17:08:29.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:29 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1174074823' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T17:08:29.602 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:30.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:30 smithi096 ceph-mon[29131]: pgmap v289: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:08:30.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:30 smithi097 ceph-mon[32602]: pgmap v289: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:08:30.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:30 smithi171 ceph-mon[33189]: pgmap v289: 97 pgs: 97 active+clean; 620 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:08:31.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 -- 172.21.15.96:0/2586405784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4103390 msgr2=0x7f3ef41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 --2- 172.21.15.96:0/2586405784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4103390 0x7f3ef41037a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00b0d0 tx=0x7f3eec00b3e0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 -- 172.21.15.96:0/2586405784 shutdown_connections 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 --2- 172.21.15.96:0/2586405784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ef4108610 0x7f3ef410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 --2- 172.21.15.96:0/2586405784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4103390 0x7f3ef41037a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 --2- 172.21.15.96:0/2586405784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4107cc0 0x7f3ef41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 -- 172.21.15.96:0/2586405784 >> 172.21.15.96:0/2586405784 conn(0x7f3ef40fec90 msgr2=0x7f3ef41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 -- 172.21.15.96:0/2586405784 shutdown_connections 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.245+0000 7f3efb32f700 1 -- 172.21.15.96:0/2586405784 wait complete. 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 Processor -- start 2024-04-25T17:08:31.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 -- start start 2024-04-25T17:08:31.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4103390 0x7f3ef419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:31.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ef4107cc0 0x7f3ef419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4108610 0x7f3ef419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4113240 con 0x7f3ef4108610 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41130c0 con 0x7f3ef4103390 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3efb32f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41133c0 con 0x7f3ef4107cc0 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3ef98cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4108610 0x7f3ef419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3ef98cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4108610 0x7f3ef419c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53374/0 (socket says 172.21.15.96:53374) 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3ef98cc700 1 -- 172.21.15.96:0/3177420136 learned_addr learned my addr 172.21.15.96:0/3177420136 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.246+0000 7f3ef90cb700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4103390 0x7f3ef419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef88ca700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ef4107cc0 0x7f3ef419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef98cc700 1 -- 172.21.15.96:0/3177420136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ef4107cc0 msgr2=0x7f3ef419c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef98cc700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ef4107cc0 0x7f3ef419c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef98cc700 1 -- 172.21.15.96:0/3177420136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4103390 msgr2=0x7f3ef419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef98cc700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4103390 0x7f3ef419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef98cc700 1 -- 172.21.15.96:0/3177420136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef41a6e90 con 0x7f3ef4108610 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef88ca700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ef4107cc0 0x7f3ef419c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3ef98cc700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4108610 0x7f3ef419c5e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d980 tx=0x7f3ef000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3eea7fc700 1 -- 172.21.15.96:0/3177420136 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0017070 con 0x7f3ef4108610 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef41a7180 con 0x7f3ef4108610 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3eea7fc700 1 -- 172.21.15.96:0/3177420136 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef0005020 con 0x7f3ef4108610 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3eea7fc700 1 -- 172.21.15.96:0/3177420136 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00099b0 con 0x7f3ef4108610 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.247+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef41a7670 con 0x7f3ef4108610 2024-04-25T17:08:31.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.248+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef4051190 con 0x7f3ef4108610 2024-04-25T17:08:31.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.250+0000 7f3eea7fc700 1 -- 172.21.15.96:0/3177420136 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f3ef0009b70 con 0x7f3ef4108610 2024-04-25T17:08:31.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.251+0000 7f3eea7fc700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3ee00afc10 0x7f3ee00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:31.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.251+0000 7f3ef90cb700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3ee00afc10 0x7f3ee00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:31.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.251+0000 7f3eea7fc700 1 -- 172.21.15.96:0/3177420136 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f3ef00dee60 con 0x7f3ef4108610 2024-04-25T17:08:31.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.252+0000 7f3ef90cb700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3ee00afc10 0x7f3ee00b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009900 tx=0x7f3eec00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:31.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.255+0000 7f3eea7fc700 1 -- 172.21.15.96:0/3177420136 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ef0020020 con 0x7f3ef4108610 2024-04-25T17:08:31.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.457+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ef402e6d0 con 0x7f3ef4108610 2024-04-25T17:08:31.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.458+0000 7f3eea7fc700 1 -- 172.21.15.96:0/3177420136 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22171 (secure 0 0 0) 0x7f3ef00a7a90 con 0x7f3ef4108610 2024-04-25T17:08:31.458 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:31.458 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":76,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:08:13.137147+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:08:31.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3ee00afc10 msgr2=0x7f3ee00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:31.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3ee00afc10 0x7f3ee00b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009900 tx=0x7f3eec00b040 comp rx=0 tx=0).stop 2024-04-25T17:08:31.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4108610 msgr2=0x7f3ef419c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:31.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4108610 0x7f3ef419c5e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d980 tx=0x7f3ef000dc90 comp rx=0 tx=0).stop 2024-04-25T17:08:31.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 shutdown_connections 2024-04-25T17:08:31.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3ee00afc10 0x7f3ee00b2120 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ef4108610 0x7f3ef419c5e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4103390 0x7f3ef419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 --2- 172.21.15.96:0/3177420136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ef4107cc0 0x7f3ef419c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:31.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 >> 172.21.15.96:0/3177420136 conn(0x7f3ef40fec90 msgr2=0x7f3ef4109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:31.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 shutdown_connections 2024-04-25T17:08:31.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:31.460+0000 7f3efb32f700 1 -- 172.21.15.96:0/3177420136 wait complete. 2024-04-25T17:08:32.164 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:32 smithi096 ceph-mon[29131]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-04-25T17:08:32.165 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:32 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3177420136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:32.199 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T17:08:32.200 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T17:08:32.445 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:32 smithi097 ceph-mon[32602]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-04-25T17:08:32.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:32 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3177420136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:32 smithi171 ceph-mon[33189]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-04-25T17:08:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:32 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3177420136' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 -- 172.21.15.96:0/639296922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 msgr2=0x7fc574111080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 --2- 172.21.15.96:0/639296922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc574111080 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00b0d0 tx=0x7fc56c00b3e0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 -- 172.21.15.96:0/639296922 shutdown_connections 2024-04-25T17:08:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 --2- 172.21.15.96:0/639296922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc574111080 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 --2- 172.21.15.96:0/639296922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741044a0 0x7fc5741048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 --2- 172.21.15.96:0/639296922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc574108dd0 0x7fc5741091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 -- 172.21.15.96:0/639296922 >> 172.21.15.96:0/639296922 conn(0x7fc5740ffd70 msgr2=0x7fc5741021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:34.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 -- 172.21.15.96:0/639296922 shutdown_connections 2024-04-25T17:08:34.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 -- 172.21.15.96:0/639296922 wait complete. 2024-04-25T17:08:34.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 Processor -- start 2024-04-25T17:08:34.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.116+0000 7fc57a26e700 1 -- start start 2024-04-25T17:08:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57a26e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741044a0 0x7fc5741a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57a26e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc574108dd0 0x7fc5741a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57a26e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc57419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57a26e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc574114370 con 0x7fc574109720 2024-04-25T17:08:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57a26e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741141f0 con 0x7fc5741044a0 2024-04-25T17:08:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57a26e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741144f0 con 0x7fc574108dd0 2024-04-25T17:08:34.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc57419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc57419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53394/0 (socket says 172.21.15.96:53394) 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 -- 172.21.15.96:0/3193900449 learned_addr learned my addr 172.21.15.96:0/3193900449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc573fff700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741044a0 0x7fc5741a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc5737fe700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc574108dd0 0x7fc5741a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 -- 172.21.15.96:0/3193900449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc574108dd0 msgr2=0x7fc5741a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc574108dd0 0x7fc5741a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 -- 172.21.15.96:0/3193900449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741044a0 msgr2=0x7fc5741a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741044a0 0x7fc5741a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 -- 172.21.15.96:0/3193900449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57419b180 con 0x7fc574109720 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc5737fe700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc574108dd0 0x7fc5741a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc573fff700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741044a0 0x7fc5741a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.117+0000 7fc57880b700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc57419aad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009850 tx=0x7fc56c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.118+0000 7fc5717fa700 1 -- 172.21.15.96:0/3193900449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c01f070 con 0x7fc574109720 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.118+0000 7fc5717fa700 1 -- 172.21.15.96:0/3193900449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56c005520 con 0x7fc574109720 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.118+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57419b410 con 0x7fc574109720 2024-04-25T17:08:34.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.118+0000 7fc5717fa700 1 -- 172.21.15.96:0/3193900449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c01b870 con 0x7fc574109720 2024-04-25T17:08:34.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.118+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57419b8f0 con 0x7fc574109720 2024-04-25T17:08:34.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.119+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc574051190 con 0x7fc574109720 2024-04-25T17:08:34.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.120+0000 7fc5717fa700 1 -- 172.21.15.96:0/3193900449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc56c022070 con 0x7fc574109720 2024-04-25T17:08:34.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.121+0000 7fc5717fa700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc55c0afc10 0x7fc55c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:34.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.121+0000 7fc573fff700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc55c0afc10 0x7fc55c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:34.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.121+0000 7fc5717fa700 1 -- 172.21.15.96:0/3193900449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fc56c00f020 con 0x7fc574109720 2024-04-25T17:08:34.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.121+0000 7fc573fff700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc55c0afc10 0x7fc55c0b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc564005950 tx=0x7fc5640058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:34.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.125+0000 7fc5717fa700 1 -- 172.21.15.96:0/3193900449 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc56c029020 con 0x7fc574109720 2024-04-25T17:08:34.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:34 smithi096 ceph-mon[29131]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T17:08:34.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.329+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc57402e510 con 0x7fc574109720 2024-04-25T17:08:34.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.331+0000 7fc5717fa700 1 -- 172.21.15.96:0/3193900449 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+22171 (secure 0 0 0) 0x7fc56c0a8af0 con 0x7fc574109720 2024-04-25T17:08:34.330 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:08:34.330 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":76,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:08:13.137147+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc55c0afc10 msgr2=0x7fc55c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc55c0afc10 0x7fc55c0b2120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc564005950 tx=0x7fc5640058e0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 msgr2=0x7fc57419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc57419aad0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009850 tx=0x7fc56c007ad0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 shutdown_connections 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc55c0afc10 0x7fc55c0b2120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc574109720 0x7fc57419aad0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5741044a0 0x7fc5741a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 --2- 172.21.15.96:0/3193900449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc574108dd0 0x7fc5741a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:34.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 >> 172.21.15.96:0/3193900449 conn(0x7fc5740ffd70 msgr2=0x7fc574108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:34.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 shutdown_connections 2024-04-25T17:08:34.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:34.333+0000 7fc57a26e700 1 -- 172.21.15.96:0/3193900449 wait complete. 2024-04-25T17:08:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:34 smithi097 ceph-mon[32602]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T17:08:34.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:34 smithi171 ceph-mon[33189]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T17:08:35.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T17:08:35.133 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T17:08:35.133 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T17:08:35.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T17:08:35.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T17:08:35.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T17:08:35.135 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T17:08:35.135 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T17:08:35.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T17:08:35.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T17:08:35.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T17:08:35.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T17:08:35.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:35 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3193900449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:35.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:35 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3193900449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:35.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:35 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3193900449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:08:35.810 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:35.817 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:35.818 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:36.279 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:36 smithi096 ceph-mon[29131]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-25T17:08:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:36 smithi097 ceph-mon[32602]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-25T17:08:36.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:36 smithi171 ceph-mon[33189]: pgmap v292: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-25T17:08:37.181 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:37.187 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:37.405 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:37 smithi096 ceph-mon[29131]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T17:08:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:37 smithi097 ceph-mon[32602]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T17:08:37.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:37 smithi171 ceph-mon[33189]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T17:08:37.617 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:37.618 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:38.092 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:39.525 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:08:40.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:40 smithi096 ceph-mon[29131]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T17:08:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:40 smithi097 ceph-mon[32602]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T17:08:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:40 smithi171 ceph-mon[33189]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-25T17:08:40.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 -- 172.21.15.96:0/306612272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281088c0 msgr2=0x7f3628108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:40.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 --2- 172.21.15.96:0/306612272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281088c0 0x7f3628108cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3618009990 tx=0x7f3618009ca0 comp rx=0 tx=0).stop 2024-04-25T17:08:40.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 -- 172.21.15.96:0/306612272 shutdown_connections 2024-04-25T17:08:40.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 --2- 172.21.15.96:0/306612272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36281031e0 0x7f362810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:40.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 --2- 172.21.15.96:0/306612272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281088c0 0x7f3628108cd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:40.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 --2- 172.21.15.96:0/306612272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3628102890 0x7f3628102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:40.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 -- 172.21.15.96:0/306612272 >> 172.21.15.96:0/306612272 conn(0x7f36280fe5c0 msgr2=0x7f36281009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:40.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 -- 172.21.15.96:0/306612272 shutdown_connections 2024-04-25T17:08:40.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.711+0000 7f362ca69700 1 -- 172.21.15.96:0/306612272 wait complete. 2024-04-25T17:08:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 Processor -- start 2024-04-25T17:08:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 -- start start 2024-04-25T17:08:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3628102890 0x7f362819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281031e0 0x7f362819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36281088c0 0x7f362819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:40.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3628112a40 con 0x7f36281031e0 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36281128c0 con 0x7f3628102890 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.712+0000 7f362ca69700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3628112bc0 con 0x7f36281088c0 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281031e0 0x7f362819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281031e0 0x7f362819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37468/0 (socket says 172.21.15.96:37468) 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 -- 172.21.15.96:0/3807624379 learned_addr learned my addr 172.21.15.96:0/3807624379 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f362659c700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3628102890 0x7f362819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3626d9d700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36281088c0 0x7f362819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 -- 172.21.15.96:0/3807624379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36281088c0 msgr2=0x7f362819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36281088c0 0x7f362819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:40.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 -- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3628102890 msgr2=0x7f362819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3628102890 0x7f362819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 -- 172.21.15.96:0/3807624379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362819a920 con 0x7f36281031e0 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f3625d9b700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281031e0 0x7f362819ed60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f361c00d950 tx=0x7f361c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c016070 con 0x7f36281031e0 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362819abb0 con 0x7f36281031e0 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f361c004830 con 0x7f36281031e0 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c005440 con 0x7f36281031e0 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.713+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36281a6730 con 0x7f36281031e0 2024-04-25T17:08:40.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.714+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3628063e20 con 0x7f36281031e0 2024-04-25T17:08:40.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.716+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f361c0055e0 con 0x7f36281031e0 2024-04-25T17:08:40.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.716+0000 7f36177fe700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f36100afc10 0x7f36100b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:40.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.716+0000 7f362659c700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f36100afc10 0x7f36100b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.716+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f361c0df040 con 0x7f36281031e0 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.717+0000 7f36177fe700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] conn(0x7f36100b9440 0x7f36100bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.717+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 --> [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f361c015ec0 con 0x7f36100b9440 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.717+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f361c0df520 con 0x7f36281031e0 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.717+0000 7f362659c700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f36100afc10 0x7f36100b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3618006010 tx=0x7f361801a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.717+0000 7f3626d9d700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] conn(0x7f36100b9440 0x7f36100bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.718+0000 7f3626d9d700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] conn(0x7f36100b9440 0x7f36100bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:40.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.718+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== osd.1 v2:172.21.15.97:6800/1211116017 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f361c015ec0 con 0x7f36100b9440 2024-04-25T17:08:40.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.734+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 --> [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3628064220 con 0x7f36100b9440 2024-04-25T17:08:40.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f36177fe700 1 -- 172.21.15.96:0/3807624379 <== osd.1 v2:172.21.15.97:6800/1211116017 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3628064220 con 0x7f36100b9440 2024-04-25T17:08:40.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] conn(0x7f36100b9440 msgr2=0x7f36100bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:40.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] conn(0x7f36100b9440 0x7f36100bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f36100afc10 msgr2=0x7f36100b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f36100afc10 0x7f36100b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3618006010 tx=0x7f361801a040 comp rx=0 tx=0).stop 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281031e0 msgr2=0x7f362819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 --2- 172.21.15.96:0/3807624379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36281031e0 0x7f362819ed60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f361c00d950 tx=0x7f361c00dc60 comp rx=0 tx=0).stop 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f3626d9d700 1 -- 172.21.15.96:0/3807624379 reap_dead start 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 shutdown_connections 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.735+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 >> 172.21.15.96:0/3807624379 conn(0x7f36280fe5c0 msgr2=0x7f3628100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.736+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 shutdown_connections 2024-04-25T17:08:40.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:40.736+0000 7f362ca69700 1 -- 172.21.15.96:0/3807624379 wait complete. 2024-04-25T17:08:42.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:42 smithi096 ceph-mon[29131]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T17:08:42.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:42 smithi097 ceph-mon[32602]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T17:08:42.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:42 smithi171 ceph-mon[33189]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 3 op/s 2024-04-25T17:08:43.823 INFO:teuthology.orchestra.run.smithi096.stdout:55834574919 2024-04-25T17:08:43.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T17:08:44.379 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:44 smithi096 ceph-mon[29131]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:44.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:44 smithi097 ceph-mon[32602]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:44.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:44 smithi171 ceph-mon[33189]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:46.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:46 smithi096 ceph-mon[29131]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:08:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:46 smithi097 ceph-mon[32602]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:08:46.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:46 smithi171 ceph-mon[33189]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:08:48.381 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:48 smithi096 ceph-mon[29131]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:48 smithi097 ceph-mon[32602]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:48 smithi171 ceph-mon[33189]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:48.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 -- 172.21.15.96:0/2954861412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 msgr2=0x7f27d010ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 --2- 172.21.15.96:0/2954861412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 0x7f27d010ff60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f27c800b0d0 tx=0x7f27c800b3e0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 -- 172.21.15.96:0/2954861412 shutdown_connections 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 --2- 172.21.15.96:0/2954861412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 0x7f27d010ff60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 --2- 172.21.15.96:0/2954861412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d0103380 0x7f27d0103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 --2- 172.21.15.96:0/2954861412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27d0107cb0 0x7f27d01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 -- 172.21.15.96:0/2954861412 >> 172.21.15.96:0/2954861412 conn(0x7f27d00fecb0 msgr2=0x7f27d01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 -- 172.21.15.96:0/2954861412 shutdown_connections 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 -- 172.21.15.96:0/2954861412 wait complete. 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.057+0000 7f27d648b700 1 Processor -- start 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d648b700 1 -- start start 2024-04-25T17:08:48.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d648b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d0103380 0x7f27d01a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d648b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27d0107cb0 0x7f27d01a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d648b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 0x7f27d01a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d648b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0113230 con 0x7f27d0108600 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d648b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01130b0 con 0x7f27d0107cb0 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d648b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01133b0 con 0x7f27d0103380 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 0x7f27d01a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 0x7f27d01a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37492/0 (socket says 172.21.15.96:37492) 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 -- 172.21.15.96:0/3698608709 learned_addr learned my addr 172.21.15.96:0/3698608709 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27cffff700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d0103380 0x7f27d01a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27cf7fe700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27d0107cb0 0x7f27d01a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 -- 172.21.15.96:0/3698608709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d0103380 msgr2=0x7f27d01a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d0103380 0x7f27d01a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 -- 172.21.15.96:0/3698608709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27d0107cb0 msgr2=0x7f27d01a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:48.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27d0107cb0 0x7f27d01a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27d4a28700 1 -- 172.21.15.96:0/3698608709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d01a60e0 con 0x7f27d0108600 2024-04-25T17:08:48.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.058+0000 7f27cf7fe700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27d0107cb0 0x7f27d01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:48.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.059+0000 7f27d4a28700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 0x7f27d01a5a40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f27c80095c0 tx=0x7f27c8007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:48.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.059+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c801f070 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.059+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d01a6370 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.059+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8003c00 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.059+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c801b6a0 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.059+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d01adcf0 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.060+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27d01a68a0 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.061+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f27c8022070 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cd7fa700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f27b80afc10 0x7f27b80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cffff700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f27b80afc10 0x7f27b80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f27c800f020 con 0x7f27d0108600 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cd7fa700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] conn(0x7f27b80b9440 0x7f27b80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 --> [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27c801cec0 con 0x7f27b80b9440 2024-04-25T17:08:48.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f27c80e08c0 con 0x7f27d0108600 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cf7fe700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] conn(0x7f27b80b9440 0x7f27b80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.062+0000 7f27cffff700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f27b80afc10 0x7f27b80b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005fd0 tx=0x7f27c000a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.063+0000 7f27cf7fe700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] conn(0x7f27b80b9440 0x7f27b80bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.063+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== osd.9 v2:172.21.15.96:6826/454601194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27c801cec0 con 0x7f27b80b9440 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.080+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 --> [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27d0101630 con 0x7f27b80b9440 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.080+0000 7f27cd7fa700 1 -- 172.21.15.96:0/3698608709 <== osd.9 v2:172.21.15.96:6826/454601194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f27d0101630 con 0x7f27b80b9440 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.080+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] conn(0x7f27b80b9440 msgr2=0x7f27b80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.080+0000 7f27d648b700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] conn(0x7f27b80b9440 0x7f27b80bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f27b80afc10 msgr2=0x7f27b80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f27b80afc10 0x7f27b80b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005fd0 tx=0x7f27c000a560 comp rx=0 tx=0).stop 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 msgr2=0x7f27d01a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 --2- 172.21.15.96:0/3698608709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27d0108600 0x7f27d01a5a40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f27c80095c0 tx=0x7f27c8007ad0 comp rx=0 tx=0).stop 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d4a28700 1 -- 172.21.15.96:0/3698608709 reap_dead start 2024-04-25T17:08:48.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 shutdown_connections 2024-04-25T17:08:48.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 >> 172.21.15.96:0/3698608709 conn(0x7f27d00fecb0 msgr2=0x7f27d0107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:48.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 shutdown_connections 2024-04-25T17:08:48.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:47.081+0000 7f27d648b700 1 -- 172.21.15.96:0/3698608709 wait complete. 2024-04-25T17:08:49.614 INFO:teuthology.orchestra.run.smithi096.stdout:257698037791 2024-04-25T17:08:49.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T17:08:50.336 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:50 smithi096 ceph-mon[29131]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 -- 172.21.15.96:0/1510839412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc1041d0 msgr2=0x7fdbdc10feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 --2- 172.21.15.96:0/1510839412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc1041d0 0x7fdbdc10feb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009990 tx=0x7fdbd4009ca0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 -- 172.21.15.96:0/1510839412 shutdown_connections 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 --2- 172.21.15.96:0/1510839412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc1041d0 0x7fdbdc10feb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 --2- 172.21.15.96:0/1510839412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbdc109170 0x7fdbdc109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 --2- 172.21.15.96:0/1510839412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbdc103880 0x7fdbdc103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 -- 172.21.15.96:0/1510839412 >> 172.21.15.96:0/1510839412 conn(0x7fdbdc0fe3f0 msgr2=0x7fdbdc100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 -- 172.21.15.96:0/1510839412 shutdown_connections 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.336+0000 7fdbe1441700 1 -- 172.21.15.96:0/1510839412 wait complete. 2024-04-25T17:08:50.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.337+0000 7fdbe1441700 1 Processor -- start 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.337+0000 7fdbe1441700 1 -- start start 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbe1441700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbdc103880 0x7fdbdc1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbe1441700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbdc1041d0 0x7fdbdc1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbe1441700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc109170 0x7fdbdc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbe1441700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc114380 con 0x7fdbdc109170 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbe1441700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc114200 con 0x7fdbdc1041d0 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbe1441700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc114500 con 0x7fdbdc103880 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc109170 0x7fdbdc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc109170 0x7fdbdc19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56140/0 (socket says 172.21.15.96:56140) 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 -- 172.21.15.96:0/810795358 learned_addr learned my addr 172.21.15.96:0/810795358 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdaffd700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbdc103880 0x7fdbdc1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbda7fc700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbdc1041d0 0x7fdbdc1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:50.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 -- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbdc103880 msgr2=0x7fdbdc1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbdc103880 0x7fdbdc1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 -- 172.21.15.96:0/810795358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbdc1041d0 msgr2=0x7fdbdc1a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbdc1041d0 0x7fdbdc1a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 -- 172.21.15.96:0/810795358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc19b1c0 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbdb7fe700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc109170 0x7fdbdc19aae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40047b0 tx=0x7fdbd4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.338+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4004830 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.339+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc19b450 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.339+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd4005020 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.339+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4003df0 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.339+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc19b920 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.339+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdbdc063e20 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.341+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fdbd40054d0 con 0x7fdbdc109170 2024-04-25T17:08:50.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.341+0000 7fdbd3fff700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdbc80afc10 0x7fdbc80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:50.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.342+0000 7fdbdaffd700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdbc80afc10 0x7fdbc80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:50.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.342+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fdbd40e0560 con 0x7fdbdc109170 2024-04-25T17:08:50.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.342+0000 7fdbd3fff700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] conn(0x7fdbc80b9440 0x7fdbc80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:50.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.342+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 --> [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdbc80bbf60 con 0x7fdbc80b9440 2024-04-25T17:08:50.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.342+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fdbd40e0a40 con 0x7fdbdc109170 2024-04-25T17:08:50.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.342+0000 7fdbdaffd700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdbc80afc10 0x7fdbc80b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc19bc20 tx=0x7fdbcc0094f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:50.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.342+0000 7fdbda7fc700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] conn(0x7fdbc80b9440 0x7fdbc80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.343+0000 7fdbda7fc700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] conn(0x7fdbc80b9440 0x7fdbc80bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.344+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== osd.8 v2:172.21.15.171:6816/2677669720 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdbc80bbf60 con 0x7fdbc80b9440 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.359+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 --> [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdbdc064220 con 0x7fdbc80b9440 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.360+0000 7fdbd3fff700 1 -- 172.21.15.96:0/810795358 <== osd.8 v2:172.21.15.171:6816/2677669720 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdbdc064220 con 0x7fdbc80b9440 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] conn(0x7fdbc80b9440 msgr2=0x7fdbc80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] conn(0x7fdbc80b9440 0x7fdbc80bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdbc80afc10 msgr2=0x7fdbc80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdbc80afc10 0x7fdbc80b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc19bc20 tx=0x7fdbcc0094f0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc109170 msgr2=0x7fdbdc19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 --2- 172.21.15.96:0/810795358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc109170 0x7fdbdc19aae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40047b0 tx=0x7fdbd4009ee0 comp rx=0 tx=0).stop 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbdb7fe700 1 -- 172.21.15.96:0/810795358 reap_dead start 2024-04-25T17:08:50.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 shutdown_connections 2024-04-25T17:08:50.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 >> 172.21.15.96:0/810795358 conn(0x7fdbdc0fe3f0 msgr2=0x7fdbdc100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:50.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 shutdown_connections 2024-04-25T17:08:50.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:50.361+0000 7fdbe1441700 1 -- 172.21.15.96:0/810795358 wait complete. 2024-04-25T17:08:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:50 smithi097 ceph-mon[32602]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:50 smithi171 ceph-mon[33189]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:08:51.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 -- 172.21.15.96:0/4069924901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 msgr2=0x7fe53410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 --2- 172.21.15.96:0/4069924901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 0x7fe53410e1d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe524009990 tx=0x7fe524009ca0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 -- 172.21.15.96:0/4069924901 shutdown_connections 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 --2- 172.21.15.96:0/4069924901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe53410e7a0 0x7fe534110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 --2- 172.21.15.96:0/4069924901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe534105330 0x7fe534105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 --2- 172.21.15.96:0/4069924901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 0x7fe53410e1d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 -- 172.21.15.96:0/4069924901 >> 172.21.15.96:0/4069924901 conn(0x7fe5340fec90 msgr2=0x7fe5341010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 -- 172.21.15.96:0/4069924901 shutdown_connections 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.239+0000 7fe53c21b700 1 -- 172.21.15.96:0/4069924901 wait complete. 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 Processor -- start 2024-04-25T17:08:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 -- start start 2024-04-25T17:08:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe534105330 0x7fe5341a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 0x7fe5341a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe53410e7a0 0x7fe53419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe534114300 con 0x7fe534105c80 2024-04-25T17:08:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe534114180 con 0x7fe53410e7a0 2024-04-25T17:08:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe53c21b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe534114480 con 0x7fe534105330 2024-04-25T17:08:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe5397b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 0x7fe5341a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe5397b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 0x7fe5341a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56162/0 (socket says 172.21.15.96:56162) 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.240+0000 7fe5397b6700 1 -- 172.21.15.96:0/3690726413 learned_addr learned my addr 172.21.15.96:0/3690726413 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe539fb7700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe534105330 0x7fe5341a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe53a7b8700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe53410e7a0 0x7fe53419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe5397b6700 1 -- 172.21.15.96:0/3690726413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe534105330 msgr2=0x7fe5341a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe5397b6700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe534105330 0x7fe5341a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe5397b6700 1 -- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe53410e7a0 msgr2=0x7fe53419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe5397b6700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe53410e7a0 0x7fe53419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe5397b6700 1 -- 172.21.15.96:0/3690726413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53419b110 con 0x7fe534105c80 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe539fb7700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe534105330 0x7fe5341a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe5397b6700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 0x7fe5341a0d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe5240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524027070 con 0x7fe534105c80 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe524003fa0 con 0x7fe534105c80 2024-04-25T17:08:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524022db0 con 0x7fe534105c80 2024-04-25T17:08:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53419b3a0 con 0x7fe534105c80 2024-04-25T17:08:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.241+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53419b880 con 0x7fe534105c80 2024-04-25T17:08:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.242+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe534063e20 con 0x7fe534105c80 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.243+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe52402a070 con 0x7fe534105c80 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.244+0000 7fe52affd700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe5200afc90 0x7fe5200b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.244+0000 7fe539fb7700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe5200afc90 0x7fe5200b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.244+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fe5240dfea0 con 0x7fe534105c80 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.244+0000 7fe52affd700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] conn(0x7fe5200b94c0 0x7fe5200bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.244+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 --> [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe524007e80 con 0x7fe5200b94c0 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.244+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fe5240e0380 con 0x7fe534105c80 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.244+0000 7fe539fb7700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe5200afc90 0x7fe5200b21a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe530005d90 tx=0x7fe530005d00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.245+0000 7fe53a7b8700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] conn(0x7fe5200b94c0 0x7fe5200bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.245+0000 7fe53a7b8700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] conn(0x7fe5200b94c0 0x7fe5200bb910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:51.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.246+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== osd.4 v2:172.21.15.97:6808/2529478189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe524007e80 con 0x7fe5200b94c0 2024-04-25T17:08:51.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.262+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 --> [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe5340642a0 con 0x7fe5200b94c0 2024-04-25T17:08:51.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.262+0000 7fe52affd700 1 -- 172.21.15.96:0/3690726413 <== osd.4 v2:172.21.15.97:6808/2529478189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe5340642a0 con 0x7fe5200b94c0 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] conn(0x7fe5200b94c0 msgr2=0x7fe5200bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] conn(0x7fe5200b94c0 0x7fe5200bb910 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe5200afc90 msgr2=0x7fe5200b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe5200afc90 0x7fe5200b21a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe530005d90 tx=0x7fe530005d00 comp rx=0 tx=0).stop 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 msgr2=0x7fe5341a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 --2- 172.21.15.96:0/3690726413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe534105c80 0x7fe5341a0d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe5240047b0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53a7b8700 1 -- 172.21.15.96:0/3690726413 reap_dead start 2024-04-25T17:08:51.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 shutdown_connections 2024-04-25T17:08:51.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 >> 172.21.15.96:0/3690726413 conn(0x7fe5340fec90 msgr2=0x7fe5340ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:51.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 shutdown_connections 2024-04-25T17:08:51.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.263+0000 7fe53c21b700 1 -- 172.21.15.96:0/3690726413 wait complete. 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.434+0000 7f7678940700 1 -- 172.21.15.96:0/2949650949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674107cc0 msgr2=0x7f76741080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.434+0000 7f7678940700 1 --2- 172.21.15.96:0/2949650949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674107cc0 0x7f76741080d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f765c0099c0 tx=0x7f765c009cd0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.434+0000 7f7678940700 1 -- 172.21.15.96:0/2949650949 shutdown_connections 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.434+0000 7f7678940700 1 --2- 172.21.15.96:0/2949650949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7674108610 0x7f767410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.434+0000 7f7678940700 1 --2- 172.21.15.96:0/2949650949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7674103390 0x7f76741037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.434+0000 7f7678940700 1 --2- 172.21.15.96:0/2949650949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674107cc0 0x7f76741080d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.434+0000 7f7678940700 1 -- 172.21.15.96:0/2949650949 >> 172.21.15.96:0/2949650949 conn(0x7f76740fec90 msgr2=0x7f76741010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:51.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.435+0000 7f7678940700 1 -- 172.21.15.96:0/2949650949 shutdown_connections 2024-04-25T17:08:51.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.435+0000 7f7678940700 1 -- 172.21.15.96:0/2949650949 wait complete. 2024-04-25T17:08:51.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.435+0000 7f7678940700 1 Processor -- start 2024-04-25T17:08:51.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.435+0000 7f7678940700 1 -- start start 2024-04-25T17:08:51.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7678940700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674103390 0x7f767419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7678940700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7674107cc0 0x7f767419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7678940700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7674108610 0x7f767419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7678940700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7674113220 con 0x7f7674103390 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7678940700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f76741130a0 con 0x7f7674107cc0 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7678940700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76741133a0 con 0x7f7674108610 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674103390 0x7f767419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674103390 0x7f767419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56176/0 (socket says 172.21.15.96:56176) 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 -- 172.21.15.96:0/3931208338 learned_addr learned my addr 172.21.15.96:0/3931208338 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7671d9b700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7674107cc0 0x7f767419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f7672d9d700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7674108610 0x7f767419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 -- 172.21.15.96:0/3931208338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7674108610 msgr2=0x7f767419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7674108610 0x7f767419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 -- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7674107cc0 msgr2=0x7f767419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7674107cc0 0x7f767419c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 -- 172.21.15.96:0/3931208338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76741a6eb0 con 0x7f7674103390 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.436+0000 7f767259c700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674103390 0x7f767419bb40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f766400ef10 tx=0x7f766400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.437+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766400ca70 con 0x7f7674103390 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.437+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7664019070 con 0x7f7674103390 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.437+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76741a7140 con 0x7f7674103390 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.437+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664014480 con 0x7f7674103390 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.437+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76741a7650 con 0x7f7674103390 2024-04-25T17:08:51.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.438+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7674063d60 con 0x7f7674103390 2024-04-25T17:08:51.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.439+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f7664012050 con 0x7f7674103390 2024-04-25T17:08:51.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f766b7fe700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f76600afc10 0x7f76600b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f7671d9b700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f76600afc10 0x7f76600b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f76640df740 con 0x7f7674103390 2024-04-25T17:08:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f766b7fe700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] conn(0x7f76600b9440 0x7f76600bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 --> [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7664025e50 con 0x7f76600b9440 2024-04-25T17:08:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f76640dfc20 con 0x7f7674103390 2024-04-25T17:08:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f7671d9b700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f76600afc10 0x7f76600b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f765c000c00 tx=0x7f765c01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.440+0000 7f7672d9d700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] conn(0x7f76600b9440 0x7f76600bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.441+0000 7f7672d9d700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] conn(0x7f76600b9440 0x7f76600bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.442+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== osd.7 v2:172.21.15.97:6816/2873132021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7664025e50 con 0x7f76600b9440 2024-04-25T17:08:51.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.458+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 --> [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f76740641e0 con 0x7f76600b9440 2024-04-25T17:08:51.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.458+0000 7f766b7fe700 1 -- 172.21.15.96:0/3931208338 <== osd.7 v2:172.21.15.97:6816/2873132021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f76740641e0 con 0x7f76600b9440 2024-04-25T17:08:51.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.458+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] conn(0x7f76600b9440 msgr2=0x7f76600bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.458+0000 7f7678940700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] conn(0x7f76600b9440 0x7f76600bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f76600afc10 msgr2=0x7f76600b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f76600afc10 0x7f76600b2120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f765c000c00 tx=0x7f765c01a040 comp rx=0 tx=0).stop 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674103390 msgr2=0x7f767419bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 --2- 172.21.15.96:0/3931208338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7674103390 0x7f767419bb40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f766400ef10 tx=0x7f766400c3a0 comp rx=0 tx=0).stop 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7672d9d700 1 -- 172.21.15.96:0/3931208338 reap_dead start 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 shutdown_connections 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 >> 172.21.15.96:0/3931208338 conn(0x7f76740fec90 msgr2=0x7f7674109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 shutdown_connections 2024-04-25T17:08:51.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:51.459+0000 7f7678940700 1 -- 172.21.15.96:0/3931208338 wait complete. 2024-04-25T17:08:52.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:52 smithi096 ceph-mon[29131]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:08:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:52 smithi097 ceph-mon[32602]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:08:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:52 smithi171 ceph-mon[33189]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:08:52.736 INFO:teuthology.orchestra.run.smithi096.stdout:231928234020 2024-04-25T17:08:52.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T17:08:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:54 smithi096 ceph-mon[29131]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:08:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:54 smithi097 ceph-mon[32602]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:08:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:54 smithi171 ceph-mon[33189]: pgmap v301: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:08:54.832 INFO:teuthology.orchestra.run.smithi096.stdout:206158430251 2024-04-25T17:08:54.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T17:08:55.241 INFO:teuthology.orchestra.run.smithi096.stdout:133143986234 2024-04-25T17:08:55.242 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T17:08:55.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:55 smithi096 ceph-mon[29131]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:08:55.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:55 smithi097 ceph-mon[32602]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:08:55.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:55 smithi171 ceph-mon[33189]: pgmap v302: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:08:55.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 -- 172.21.15.96:0/938654041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5800105c40 msgr2=0x7f580010e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:55.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 --2- 172.21.15.96:0/938654041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5800105c40 0x7f580010e190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009990 tx=0x7f57e8009ca0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 -- 172.21.15.96:0/938654041 shutdown_connections 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 --2- 172.21.15.96:0/938654041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f580010e760 0x7f5800110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 --2- 172.21.15.96:0/938654041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58001052f0 0x7f5800105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 --2- 172.21.15.96:0/938654041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5800105c40 0x7f580010e190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 -- 172.21.15.96:0/938654041 >> 172.21.15.96:0/938654041 conn(0x7f58000fec30 msgr2=0x7f5800101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.852+0000 7f58061bc700 1 -- 172.21.15.96:0/938654041 shutdown_connections 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.853+0000 7f58061bc700 1 -- 172.21.15.96:0/938654041 wait complete. 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.853+0000 7f58061bc700 1 Processor -- start 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.853+0000 7f58061bc700 1 -- start start 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.853+0000 7f58061bc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58001052f0 0x7f58001997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.853+0000 7f58061bc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5800105c40 0x7f5800199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:55.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.853+0000 7f58061bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f580010e760 0x7f58001a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f58061bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58001142e0 con 0x7f580010e760 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f58061bc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5800114160 con 0x7f5800105c40 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f58061bc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5800114460 con 0x7f58001052f0 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f580010e760 0x7f58001a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f580010e760 0x7f58001a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56188/0 (socket says 172.21.15.96:56188) 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 -- 172.21.15.96:0/3271523205 learned_addr learned my addr 172.21.15.96:0/3271523205 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57ff7fe700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58001052f0 0x7f58001997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57f7fff700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5800105c40 0x7f5800199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 -- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58001052f0 msgr2=0x7f58001997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58001052f0 0x7f58001997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 -- 172.21.15.96:0/3271523205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5800105c40 msgr2=0x7f5800199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5800105c40 0x7f5800199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 -- 172.21.15.96:0/3271523205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001a0d50 con 0x7f580010e760 2024-04-25T17:08:55.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fffff700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f580010e760 0x7f58001a0810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f57f8007ee0 tx=0x7f57f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:55.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f800dbd0 con 0x7f580010e760 2024-04-25T17:08:55.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57f800dd70 con 0x7f580010e760 2024-04-25T17:08:55.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8005990 con 0x7f580010e760 2024-04-25T17:08:55.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001a1040 con 0x7f580010e760 2024-04-25T17:08:55.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.854+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001a1530 con 0x7f580010e760 2024-04-25T17:08:55.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.855+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f580019a610 con 0x7f580010e760 2024-04-25T17:08:55.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.857+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f57f801a020 con 0x7f580010e760 2024-04-25T17:08:55.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.857+0000 7f57fd7fa700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f57ec0afc10 0x7f57ec0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:55.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.857+0000 7f57ff7fe700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f57ec0afc10 0x7f57ec0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:55.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.857+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f57f80dea20 con 0x7f580010e760 2024-04-25T17:08:55.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.858+0000 7f57fd7fa700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] conn(0x7f57ec0b9440 0x7f57ec0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:55.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.858+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 --> [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57ec0bbf60 con 0x7f57ec0b9440 2024-04-25T17:08:55.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.858+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f57f80def00 con 0x7f580010e760 2024-04-25T17:08:55.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.858+0000 7f57ff7fe700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f57ec0afc10 0x7f57ec0b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009c60 tx=0x7f57f0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:55.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.858+0000 7f57f7fff700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] conn(0x7f57ec0b9440 0x7f57ec0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:55.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.858+0000 7f57f7fff700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] conn(0x7f57ec0b9440 0x7f57ec0bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:55.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.859+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== osd.2 v2:172.21.15.171:6800/3917754473 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec0bbf60 con 0x7f57ec0b9440 2024-04-25T17:08:55.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.875+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 --> [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5800103be0 con 0x7f57ec0b9440 2024-04-25T17:08:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f57fd7fa700 1 -- 172.21.15.96:0/3271523205 <== osd.2 v2:172.21.15.171:6800/3917754473 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5800103be0 con 0x7f57ec0b9440 2024-04-25T17:08:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] conn(0x7f57ec0b9440 msgr2=0x7f57ec0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f58061bc700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] conn(0x7f57ec0b9440 0x7f57ec0bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f57ec0afc10 msgr2=0x7f57ec0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f58061bc700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f57ec0afc10 0x7f57ec0b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009c60 tx=0x7f57f0009260 comp rx=0 tx=0).stop 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f580010e760 msgr2=0x7f58001a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f58061bc700 1 --2- 172.21.15.96:0/3271523205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f580010e760 0x7f58001a0810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f57f8007ee0 tx=0x7f57f800d3b0 comp rx=0 tx=0).stop 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.876+0000 7f57fffff700 1 -- 172.21.15.96:0/3271523205 reap_dead start 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.877+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 shutdown_connections 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.877+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 >> 172.21.15.96:0/3271523205 conn(0x7f58000fec30 msgr2=0x7f5800102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.877+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 shutdown_connections 2024-04-25T17:08:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:55.877+0000 7f58061bc700 1 -- 172.21.15.96:0/3271523205 wait complete. 2024-04-25T17:08:56.975 INFO:teuthology.orchestra.run.smithi096.stdout:77309411397 2024-04-25T17:08:56.975 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.162+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/2856912859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad0103840 msgr2=0x7f6ad0103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.162+0000 7f6ad7c4e700 1 --2- 172.21.15.96:0/2856912859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad0103840 0x7f6ad0103c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009990 tx=0x7f6ac0009ca0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/2856912859 shutdown_connections 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 --2- 172.21.15.96:0/2856912859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ad0104190 0x7f6ad010ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 --2- 172.21.15.96:0/2856912859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ad01080a0 0x7f6ad01084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 --2- 172.21.15.96:0/2856912859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad0103840 0x7f6ad0103c50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/2856912859 >> 172.21.15.96:0/2856912859 conn(0x7f6ad00fe3f0 msgr2=0x7f6ad0100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/2856912859 shutdown_connections 2024-04-25T17:08:57.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/2856912859 wait complete. 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 Processor -- start 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.163+0000 7f6ad7c4e700 1 -- start start 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad7c4e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ad0103840 0x7f6ad019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad7c4e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ad0104190 0x7f6ad019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad7c4e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad01080a0 0x7f6ad01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad7c4e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0116570 con 0x7f6ad01080a0 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad7c4e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01163f0 con 0x7f6ad0103840 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad7c4e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad01166f0 con 0x7f6ad0104190 2024-04-25T17:08:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad01080a0 0x7f6ad01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad01080a0 0x7f6ad01a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56206/0 (socket says 172.21.15.96:56206) 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 -- 172.21.15.96:0/516608697 learned_addr learned my addr 172.21.15.96:0/516608697 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad59ea700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ad0103840 0x7f6ad019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad51e9700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ad0104190 0x7f6ad019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 -- 172.21.15.96:0/516608697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ad0104190 msgr2=0x7f6ad019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ad0104190 0x7f6ad019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 -- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ad0103840 msgr2=0x7f6ad019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ad0103840 0x7f6ad019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad61eb700 1 -- 172.21.15.96:0/516608697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad01a1d70 con 0x7f6ad01080a0 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.164+0000 7f6ad59ea700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ad0103840 0x7f6ad019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.165+0000 7f6ad61eb700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad01080a0 0x7f6ad01a16d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800ef30 tx=0x7f6ac800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:57.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.165+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac80092e0 con 0x7f6ad01080a0 2024-04-25T17:08:57.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.165+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac8004bb0 con 0x7f6ad01080a0 2024-04-25T17:08:57.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.165+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad01a2000 con 0x7f6ad01080a0 2024-04-25T17:08:57.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.165+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8004280 con 0x7f6ad01080a0 2024-04-25T17:08:57.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.165+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad01a9980 con 0x7f6ad01080a0 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.166+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ad0063e20 con 0x7f6ad01080a0 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.167+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f6ac801b030 con 0x7f6ad01080a0 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.168+0000 7f6ac6ffd700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6abc0afc90 0x7f6abc0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.168+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f6ac80df530 con 0x7f6ad01080a0 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.168+0000 7f6ad59ea700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6abc0afc90 0x7f6abc0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.168+0000 7f6ac6ffd700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] conn(0x7f6abc0b94c0 0x7f6abc0bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.168+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 --> [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ac8011e50 con 0x7f6abc0b94c0 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.168+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6ac80dfa10 con 0x7f6ad01080a0 2024-04-25T17:08:57.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.168+0000 7f6ad51e9700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] conn(0x7f6abc0b94c0 0x7f6abc0bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.169+0000 7f6ad59ea700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6abc0afc90 0x7f6abc0b21a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ad006b4b0 tx=0x7f6acc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:57.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.169+0000 7f6ad51e9700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] conn(0x7f6abc0b94c0 0x7f6abc0bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:57.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.170+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== osd.10 v2:172.21.15.97:6824/213999228 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ac8011e50 con 0x7f6abc0b94c0 2024-04-25T17:08:57.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.186+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 --> [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ad0064220 con 0x7f6abc0b94c0 2024-04-25T17:08:57.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.187+0000 7f6ac6ffd700 1 -- 172.21.15.96:0/516608697 <== osd.10 v2:172.21.15.97:6824/213999228 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6ad0064220 con 0x7f6abc0b94c0 2024-04-25T17:08:57.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] conn(0x7f6abc0b94c0 msgr2=0x7f6abc0bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] conn(0x7f6abc0b94c0 0x7f6abc0bb910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6abc0afc90 msgr2=0x7f6abc0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6abc0afc90 0x7f6abc0b21a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ad006b4b0 tx=0x7f6acc009260 comp rx=0 tx=0).stop 2024-04-25T17:08:57.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad01080a0 msgr2=0x7f6ad01a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 --2- 172.21.15.96:0/516608697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad01080a0 0x7f6ad01a16d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800ef30 tx=0x7f6ac800c450 comp rx=0 tx=0).stop 2024-04-25T17:08:57.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad61eb700 1 -- 172.21.15.96:0/516608697 reap_dead start 2024-04-25T17:08:57.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 shutdown_connections 2024-04-25T17:08:57.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 >> 172.21.15.96:0/516608697 conn(0x7f6ad00fe3f0 msgr2=0x7f6ad0109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:57.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 shutdown_connections 2024-04-25T17:08:57.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.188+0000 7f6ad7c4e700 1 -- 172.21.15.96:0/516608697 wait complete. 2024-04-25T17:08:57.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 -- 172.21.15.96:0/757735999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c108de0 msgr2=0x7fb47c1091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 --2- 172.21.15.96:0/757735999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c108de0 0x7fb47c1091f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4640099c0 tx=0x7fb464009cd0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 -- 172.21.15.96:0/757735999 shutdown_connections 2024-04-25T17:08:57.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 --2- 172.21.15.96:0/757735999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb47c109730 0x7fb47c111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 --2- 172.21.15.96:0/757735999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb47c1029c0 0x7fb47c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 --2- 172.21.15.96:0/757735999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c108de0 0x7fb47c1091f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 -- 172.21.15.96:0/757735999 >> 172.21.15.96:0/757735999 conn(0x7fb47c0fe3f0 msgr2=0x7fb47c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 -- 172.21.15.96:0/757735999 shutdown_connections 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.472+0000 7fb48113d700 1 -- 172.21.15.96:0/757735999 wait complete. 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.473+0000 7fb48113d700 1 Processor -- start 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.473+0000 7fb48113d700 1 -- start start 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.473+0000 7fb48113d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c1029c0 0x7fb47c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.473+0000 7fb48113d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb47c108de0 0x7fb47c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.473+0000 7fb48113d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb47c109730 0x7fb47c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb48113d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c114350 con 0x7fb47c1029c0 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb48113d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c1141d0 con 0x7fb47c108de0 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb48113d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c1144d0 con 0x7fb47c109730 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c1029c0 0x7fb47c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c1029c0 0x7fb47c19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47846/0 (socket says 172.21.15.96:47846) 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 -- 172.21.15.96:0/2425665349 learned_addr learned my addr 172.21.15.96:0/2425665349 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47259c700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb47c108de0 0x7fb47c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47b59e700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb47c109730 0x7fb47c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 -- 172.21.15.96:0/2425665349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb47c109730 msgr2=0x7fb47c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb47c109730 0x7fb47c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 -- 172.21.15.96:0/2425665349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb47c108de0 msgr2=0x7fb47c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb47c108de0 0x7fb47c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 -- 172.21.15.96:0/2425665349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c19fbe0 con 0x7fb47c1029c0 2024-04-25T17:08:57.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb47ad9d700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c1029c0 0x7fb47c19ad60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00e950 tx=0x7fb46c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:57.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c00eea0 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c19fed0 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb46c0044e0 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c003e00 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.474+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a77f0 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.475+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb47c063e20 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.477+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb46c003fa0 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.477+0000 7fb478d99700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb4680afc10 0x7fb4680b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb47259c700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb4680afc10 0x7fb4680b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fb46c0df560 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb478d99700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] conn(0x7fb4680b9440 0x7fb4680bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 --> [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4680bbf60 con 0x7fb4680b9440 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb46c0dfa40 con 0x7fb47c1029c0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb47259c700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb4680afc10 0x7fb4680b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb464000c00 tx=0x7fb46401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:57.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb47b59e700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] conn(0x7fb4680b9440 0x7fb4680bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.478+0000 7fb47b59e700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] conn(0x7fb4680b9440 0x7fb4680bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.479+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== osd.3 v2:172.21.15.96:6810/3604096933 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4680bbf60 con 0x7fb4680b9440 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 --> [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb47c064220 con 0x7fb4680b9440 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb478d99700 1 -- 172.21.15.96:0/2425665349 <== osd.3 v2:172.21.15.96:6810/3604096933 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb47c064220 con 0x7fb4680b9440 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] conn(0x7fb4680b9440 msgr2=0x7fb4680bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb48113d700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] conn(0x7fb4680b9440 0x7fb4680bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb4680afc10 msgr2=0x7fb4680b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb48113d700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb4680afc10 0x7fb4680b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb464000c00 tx=0x7fb46401a040 comp rx=0 tx=0).stop 2024-04-25T17:08:57.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c1029c0 msgr2=0x7fb47c19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:57.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb48113d700 1 --2- 172.21.15.96:0/2425665349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47c1029c0 0x7fb47c19ad60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00e950 tx=0x7fb46c00ec60 comp rx=0 tx=0).stop 2024-04-25T17:08:57.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.495+0000 7fb47b59e700 1 -- 172.21.15.96:0/2425665349 reap_dead start 2024-04-25T17:08:57.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.496+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 shutdown_connections 2024-04-25T17:08:57.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.496+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 >> 172.21.15.96:0/2425665349 conn(0x7fb47c0fe3f0 msgr2=0x7fb47c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:57.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.496+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 shutdown_connections 2024-04-25T17:08:57.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:57.496+0000 7fb48113d700 1 -- 172.21.15.96:0/2425665349 wait complete. 2024-04-25T17:08:58.343 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:08:58 smithi096 ceph-mon[29131]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:08:58.373 INFO:teuthology.orchestra.run.smithi096.stdout:107374182465 2024-04-25T17:08:58.373 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T17:08:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:08:58 smithi097 ceph-mon[32602]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:08:58.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:08:58 smithi171 ceph-mon[33189]: pgmap v303: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:08:58.755 INFO:teuthology.orchestra.run.smithi096.stdout:279172874267 2024-04-25T17:08:58.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 -- 172.21.15.96:0/1886193759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a18105cb0 msgr2=0x7f1a1810e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 --2- 172.21.15.96:0/1886193759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a18105cb0 0x7f1a1810e200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009990 tx=0x7f1a04009ca0 comp rx=0 tx=0).stop 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 -- 172.21.15.96:0/1886193759 shutdown_connections 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 --2- 172.21.15.96:0/1886193759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a1810e7d0 0x7f1a18110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 --2- 172.21.15.96:0/1886193759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a18105360 0x7f1a18105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 --2- 172.21.15.96:0/1886193759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a18105cb0 0x7f1a1810e200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 -- 172.21.15.96:0/1886193759 >> 172.21.15.96:0/1886193759 conn(0x7f1a180fedb0 msgr2=0x7f1a181011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 -- 172.21.15.96:0/1886193759 shutdown_connections 2024-04-25T17:08:58.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 -- 172.21.15.96:0/1886193759 wait complete. 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 Processor -- start 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.996+0000 7f1a1f114700 1 -- start start 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1f114700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a18105360 0x7f1a1819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1f114700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18105cb0 0x7f1a1819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1f114700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a1810e7d0 0x7f1a1819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1f114700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a18114420 con 0x7f1a1810e7d0 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1f114700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181142a0 con 0x7f1a18105cb0 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1f114700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181145a0 con 0x7f1a18105360 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a1810e7d0 0x7f1a1819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:58.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a1810e7d0 0x7f1a1819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47864/0 (socket says 172.21.15.96:47864) 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 -- 172.21.15.96:0/3594324263 learned_addr learned my addr 172.21.15.96:0/3594324263 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a17fff700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18105cb0 0x7f1a1819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1ceb0700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a18105360 0x7f1a1819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 -- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a18105360 msgr2=0x7f1a1819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a18105360 0x7f1a1819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 -- 172.21.15.96:0/3594324263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18105cb0 msgr2=0x7f1a1819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18105cb0 0x7f1a1819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 -- 172.21.15.96:0/3594324263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1819fd80 con 0x7f1a1810e7d0 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1ceb0700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a18105360 0x7f1a1819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.997+0000 7f1a1d6b1700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a1810e7d0 0x7f1a1819f6e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000b970 tx=0x7f1a1000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.998+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10019070 con 0x7f1a1810e7d0 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.998+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1000cde0 con 0x7f1a1810e7d0 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.998+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a181a0070 con 0x7f1a1810e7d0 2024-04-25T17:08:58.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.998+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10007370 con 0x7f1a1810e7d0 2024-04-25T17:08:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.998+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a181a7990 con 0x7f1a1810e7d0 2024-04-25T17:08:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:58.999+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a18063da0 con 0x7f1a1810e7d0 2024-04-25T17:08:59.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.000+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1a1001c070 con 0x7f1a1810e7d0 2024-04-25T17:08:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.001+0000 7f1a15ffb700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a080afc10 0x7f1a080b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.001+0000 7f1a1ceb0700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a080afc10 0x7f1a080b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.001+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f1a100ded80 con 0x7f1a1810e7d0 2024-04-25T17:08:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.001+0000 7f1a15ffb700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] conn(0x7f1a080b9440 0x7f1a080bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.001+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 --> [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a080bbf60 con 0x7f1a080b9440 2024-04-25T17:08:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.001+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1a100df260 con 0x7f1a1810e7d0 2024-04-25T17:08:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.001+0000 7f1a1ceb0700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a080afc10 0x7f1a080b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007900 tx=0x7f1a0c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.002+0000 7f1a17fff700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] conn(0x7f1a080b9440 0x7f1a080bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:59.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.002+0000 7f1a17fff700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] conn(0x7f1a080b9440 0x7f1a080bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:59.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.003+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== osd.5 v2:172.21.15.171:6808/651283104 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a080bbf60 con 0x7f1a080b9440 2024-04-25T17:08:59.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.019+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 --> [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a180641d0 con 0x7f1a080b9440 2024-04-25T17:08:59.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a15ffb700 1 -- 172.21.15.96:0/3594324263 <== osd.5 v2:172.21.15.171:6808/651283104 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a180641d0 con 0x7f1a080b9440 2024-04-25T17:08:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] conn(0x7f1a080b9440 msgr2=0x7f1a080bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] conn(0x7f1a080b9440 0x7f1a080bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a080afc10 msgr2=0x7f1a080b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f1a080afc10 0x7f1a080b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007900 tx=0x7f1a0c008040 comp rx=0 tx=0).stop 2024-04-25T17:08:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a1810e7d0 msgr2=0x7f1a1819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 --2- 172.21.15.96:0/3594324263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a1810e7d0 0x7f1a1819f6e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000b970 tx=0x7f1a1000bc80 comp rx=0 tx=0).stop 2024-04-25T17:08:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1d6b1700 1 -- 172.21.15.96:0/3594324263 reap_dead start 2024-04-25T17:08:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 shutdown_connections 2024-04-25T17:08:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 >> 172.21.15.96:0/3594324263 conn(0x7f1a180fedb0 msgr2=0x7f1a180ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 shutdown_connections 2024-04-25T17:08:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.020+0000 7f1a1f114700 1 -- 172.21.15.96:0/3594324263 wait complete. 2024-04-25T17:08:59.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 -- 172.21.15.96:0/558056608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2138103350 msgr2=0x7f2138103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 --2- 172.21.15.96:0/558056608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2138103350 0x7f2138103760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21340099c0 tx=0x7f2134009cd0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 -- 172.21.15.96:0/558056608 shutdown_connections 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 --2- 172.21.15.96:0/558056608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21381085d0 0x7f213810ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 --2- 172.21.15.96:0/558056608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2138103350 0x7f2138103760 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 --2- 172.21.15.96:0/558056608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2138107c80 0x7f2138108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 -- 172.21.15.96:0/558056608 >> 172.21.15.96:0/558056608 conn(0x7f21380fec30 msgr2=0x7f2138101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 -- 172.21.15.96:0/558056608 shutdown_connections 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.791+0000 7f213f497700 1 -- 172.21.15.96:0/558056608 wait complete. 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 Processor -- start 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 -- start start 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2138103350 0x7f213819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2138107c80 0x7f213819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21381085d0 0x7f2138199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2138113250 con 0x7f21381085d0 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f21381130d0 con 0x7f2138103350 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213f497700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f21381133d0 con 0x7f2138107c80 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213da34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21381085d0 0x7f2138199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213da34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21381085d0 0x7f2138199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47880/0 (socket says 172.21.15.96:47880) 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213da34700 1 -- 172.21.15.96:0/2723091241 learned_addr learned my addr 172.21.15.96:0/2723091241 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213ca32700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2138107c80 0x7f213819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.792+0000 7f213d233700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2138103350 0x7f213819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213da34700 1 -- 172.21.15.96:0/2723091241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2138107c80 msgr2=0x7f213819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213da34700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2138107c80 0x7f213819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213da34700 1 -- 172.21.15.96:0/2723091241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2138103350 msgr2=0x7f213819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213da34700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2138103350 0x7f213819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213da34700 1 -- 172.21.15.96:0/2723091241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213819a140 con 0x7f21381085d0 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213d233700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2138103350 0x7f213819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:08:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213da34700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21381085d0 0x7f2138199a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2130007f10 tx=0x7f213000f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130016030 con 0x7f21381085d0 2024-04-25T17:08:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213819a430 con 0x7f21381085d0 2024-04-25T17:08:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f213000fde0 con 0x7f21381085d0 2024-04-25T17:08:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130005020 con 0x7f21381085d0 2024-04-25T17:08:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.793+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213806bf10 con 0x7f21381085d0 2024-04-25T17:08:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.794+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f213806c460 con 0x7f21381085d0 2024-04-25T17:08:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.796+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f2130004cb0 con 0x7f21381085d0 2024-04-25T17:08:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.797+0000 7f212e7fc700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f21240afc10 0x7f21240b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.797+0000 7f213d233700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f21240afc10 0x7f21240b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.797+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f21300e1250 con 0x7f21381085d0 2024-04-25T17:08:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.797+0000 7f212e7fc700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] conn(0x7f21240b9440 0x7f21240bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:08:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.797+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 --> [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21240bbf60 con 0x7f21240b9440 2024-04-25T17:08:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.797+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f21300e1730 con 0x7f21381085d0 2024-04-25T17:08:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.798+0000 7f213ca32700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] conn(0x7f21240b9440 0x7f21240bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:08:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.798+0000 7f213d233700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f21240afc10 0x7f21240b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2134000c00 tx=0x7f213401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.798+0000 7f213ca32700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] conn(0x7f21240b9440 0x7f21240bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:08:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.799+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== osd.0 v2:172.21.15.96:6802/1323373413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21240bbf60 con 0x7f21240b9440 2024-04-25T17:08:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.815+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 --> [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f213806c8e0 con 0x7f21240b9440 2024-04-25T17:08:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.815+0000 7f212e7fc700 1 -- 172.21.15.96:0/2723091241 <== osd.0 v2:172.21.15.96:6802/1323373413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f213806c8e0 con 0x7f21240b9440 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] conn(0x7f21240b9440 msgr2=0x7f21240bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] conn(0x7f21240b9440 0x7f21240bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f21240afc10 msgr2=0x7f21240b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f21240afc10 0x7f21240b2120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2134000c00 tx=0x7f213401a040 comp rx=0 tx=0).stop 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21381085d0 msgr2=0x7f2138199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 --2- 172.21.15.96:0/2723091241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21381085d0 0x7f2138199a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2130007f10 tx=0x7f213000f5c0 comp rx=0 tx=0).stop 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213da34700 1 -- 172.21.15.96:0/2723091241 reap_dead start 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 shutdown_connections 2024-04-25T17:08:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 >> 172.21.15.96:0/2723091241 conn(0x7f21380fec30 msgr2=0x7f2138101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:08:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 shutdown_connections 2024-04-25T17:08:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:08:59.816+0000 7f213f497700 1 -- 172.21.15.96:0/2723091241 wait complete. 2024-04-25T17:09:00.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:00 smithi096 ceph-mon[29131]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T17:09:00.429 INFO:teuthology.orchestra.run.smithi096.stdout:154618822711 2024-04-25T17:09:00.430 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T17:09:00.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:00 smithi097 ceph-mon[32602]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T17:09:00.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:00 smithi171 ceph-mon[33189]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T17:09:00.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 -- 172.21.15.96:0/3994642042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc01085e0 msgr2=0x7fddc010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:00.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 --2- 172.21.15.96:0/3994642042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc01085e0 0x7fddc010ff40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fddb80099c0 tx=0x7fddb8009cd0 comp rx=0 tx=0).stop 2024-04-25T17:09:00.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 -- 172.21.15.96:0/3994642042 shutdown_connections 2024-04-25T17:09:00.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 --2- 172.21.15.96:0/3994642042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc01085e0 0x7fddc010ff40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:00.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 --2- 172.21.15.96:0/3994642042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc0103360 0x7fddc0103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 --2- 172.21.15.96:0/3994642042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddc0107c90 0x7fddc01080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 -- 172.21.15.96:0/3994642042 >> 172.21.15.96:0/3994642042 conn(0x7fddc00fec90 msgr2=0x7fddc01010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 -- 172.21.15.96:0/3994642042 shutdown_connections 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.877+0000 7fddc6b5b700 1 -- 172.21.15.96:0/3994642042 wait complete. 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 Processor -- start 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 -- start start 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc0103360 0x7fddc019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddc0107c90 0x7fddc019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc01085e0 0x7fddc01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0113290 con 0x7fddc0103360 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0113110 con 0x7fddc01085e0 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc6b5b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0113410 con 0x7fddc0107c90 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc48f7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc0103360 0x7fddc019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc48f7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc0103360 0x7fddc019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47900/0 (socket says 172.21.15.96:47900) 2024-04-25T17:09:00.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc48f7700 1 -- 172.21.15.96:0/283406825 learned_addr learned my addr 172.21.15.96:0/283406825 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddb7fff700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddc0107c90 0x7fddc019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.878+0000 7fddc50f8700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc01085e0 0x7fddc01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc48f7700 1 -- 172.21.15.96:0/283406825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddc0107c90 msgr2=0x7fddc019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc48f7700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fddc0107c90 0x7fddc019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc48f7700 1 -- 172.21.15.96:0/283406825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc01085e0 msgr2=0x7fddc01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc48f7700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc01085e0 0x7fddc01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc48f7700 1 -- 172.21.15.96:0/283406825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc019a080 con 0x7fddc0103360 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc50f8700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc01085e0 0x7fddc01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc48f7700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc0103360 0x7fddc019f7b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00ba50 tx=0x7fddbc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc019070 con 0x7fddc0103360 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddbc014440 con 0x7fddc0103360 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc019a370 con 0x7fddc0103360 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00ce60 con 0x7fddc0103360 2024-04-25T17:09:00.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.879+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc006b4e0 con 0x7fddc0103360 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.880+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fddc006baf0 con 0x7fddc0103360 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.881+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fddbc01c070 con 0x7fddc0103360 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddb5ffb700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdda80afc10 0x7fdda80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddb7fff700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdda80afc10 0x7fdda80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7fddbc0de750 con 0x7fddc0103360 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddb5ffb700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] conn(0x7fdda80b9440 0x7fdda80bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 --> [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdda80bbf60 con 0x7fdda80b9440 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fddbc0dec30 con 0x7fddc0103360 2024-04-25T17:09:00.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddc50f8700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] conn(0x7fdda80b9440 0x7fdda80bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:00.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.882+0000 7fddb7fff700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdda80afc10 0x7fdda80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fddc0102800 tx=0x7fddb000a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:00.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.883+0000 7fddc50f8700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] conn(0x7fdda80b9440 0x7fdda80bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:00.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.884+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== osd.6 v2:172.21.15.96:6818/2945403945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdda80bbf60 con 0x7fdda80b9440 2024-04-25T17:09:00.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.900+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 --> [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fddc006bf70 con 0x7fdda80b9440 2024-04-25T17:09:00.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.900+0000 7fddb5ffb700 1 -- 172.21.15.96:0/283406825 <== osd.6 v2:172.21.15.96:6818/2945403945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fddc006bf70 con 0x7fdda80b9440 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] conn(0x7fdda80b9440 msgr2=0x7fdda80bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] conn(0x7fdda80b9440 0x7fdda80bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdda80afc10 msgr2=0x7fdda80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdda80afc10 0x7fdda80b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fddc0102800 tx=0x7fddb000a300 comp rx=0 tx=0).stop 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc0103360 msgr2=0x7fddc019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 --2- 172.21.15.96:0/283406825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc0103360 0x7fddc019f7b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00ba50 tx=0x7fddbc00bd60 comp rx=0 tx=0).stop 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc50f8700 1 -- 172.21.15.96:0/283406825 reap_dead start 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 shutdown_connections 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 >> 172.21.15.96:0/283406825 conn(0x7fddc00fec90 msgr2=0x7fddc0109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 shutdown_connections 2024-04-25T17:09:00.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:00.901+0000 7fddc6b5b700 1 -- 172.21.15.96:0/283406825 wait complete. 2024-04-25T17:09:01.542 INFO:teuthology.orchestra.run.smithi096.stdout:34359738448 2024-04-25T17:09:01.542 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T17:09:02.278 INFO:teuthology.orchestra.run.smithi096.stdout:180388626482 2024-04-25T17:09:02.278 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T17:09:02.333 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:02.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:02 smithi096 ceph-mon[29131]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:02 smithi097 ceph-mon[32602]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:02.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:02 smithi171 ceph-mon[33189]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.579+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1813370601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b81044a0 msgr2=0x7f92b81048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.579+0000 7f92bf3ab700 1 --2- 172.21.15.96:0/1813370601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b81044a0 0x7f92b81048b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f92b40099c0 tx=0x7f92b4009cd0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1813370601 shutdown_connections 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 --2- 172.21.15.96:0/1813370601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b8109720 0x7f92b8111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 --2- 172.21.15.96:0/1813370601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b81044a0 0x7f92b81048b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 --2- 172.21.15.96:0/1813370601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8108dd0 0x7f92b81091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1813370601 >> 172.21.15.96:0/1813370601 conn(0x7f92b80ffeb0 msgr2=0x7f92b81022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1813370601 shutdown_connections 2024-04-25T17:09:02.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1813370601 wait complete. 2024-04-25T17:09:02.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 Processor -- start 2024-04-25T17:09:02.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.580+0000 7f92bf3ab700 1 -- start start 2024-04-25T17:09:02.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b81044a0 0x7f92b81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:02.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b8108dd0 0x7f92b81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8109720 0x7f92b819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8114350 con 0x7f92b8108dd0 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81141d0 con 0x7f92b81044a0 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81144d0 con 0x7f92b8109720 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b8108dd0 0x7f92b81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b8108dd0 0x7f92b81a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47926/0 (socket says 172.21.15.96:47926) 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 -- 172.21.15.96:0/1495527701 learned_addr learned my addr 172.21.15.96:0/1495527701 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bd147700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b81044a0 0x7f92b81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bd948700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8109720 0x7f92b819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 -- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8109720 msgr2=0x7f92b819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b8109720 0x7f92b819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 -- 172.21.15.96:0/1495527701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b81044a0 msgr2=0x7f92b81a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b81044a0 0x7f92b81a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 -- 172.21.15.96:0/1495527701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b819b0f0 con 0x7f92b8108dd0 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bd147700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92b81044a0 0x7f92b81a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bc946700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b8108dd0 0x7f92b81a0dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f92a800c9e0 tx=0x7f92a800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8007aa0 con 0x7f92b8108dd0 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92a8007c40 con 0x7f92b8108dd0 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8018660 con 0x7f92b8108dd0 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b819b3e0 con 0x7f92b8108dd0 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.581+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b81a7720 con 0x7f92b8108dd0 2024-04-25T17:09:02.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.582+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92b8063e20 con 0x7f92b8108dd0 2024-04-25T17:09:02.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.584+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f92a8018800 con 0x7f92b8108dd0 2024-04-25T17:09:02.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.584+0000 7f92ae7fc700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92a40afc90 0x7f92a40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:02.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.584+0000 7f92bd147700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92a40afc90 0x7f92a40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.585+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7979+0+0 (secure 0 0 0) 0x7f92a80deee0 con 0x7f92b8108dd0 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.585+0000 7f92ae7fc700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] conn(0x7f92a40b94c0 0x7f92a40bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.585+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 --> [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92a8015e70 con 0x7f92a40b94c0 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.585+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f92a80df3c0 con 0x7f92b8108dd0 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.585+0000 7f92bd147700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92a40afc90 0x7f92a40b21a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f92b4000c00 tx=0x7f92b401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.585+0000 7f92bd948700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] conn(0x7f92a40b94c0 0x7f92a40bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.585+0000 7f92bd948700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] conn(0x7f92a40b94c0 0x7f92a40bb910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:02.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.586+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== osd.11 v2:172.21.15.171:6824/2756841821 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92a8015e70 con 0x7f92a40b94c0 2024-04-25T17:09:02.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.602+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 --> [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92b8064220 con 0x7f92a40b94c0 2024-04-25T17:09:02.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.603+0000 7f92ae7fc700 1 -- 172.21.15.96:0/1495527701 <== osd.11 v2:172.21.15.171:6824/2756841821 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f92b8064220 con 0x7f92a40b94c0 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.603+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] conn(0x7f92a40b94c0 msgr2=0x7f92a40bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.603+0000 7f92bf3ab700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] conn(0x7f92a40b94c0 0x7f92a40bb910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92a40afc90 msgr2=0x7f92a40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f92a40afc90 0x7f92a40b21a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f92b4000c00 tx=0x7f92b401a040 comp rx=0 tx=0).stop 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b8108dd0 msgr2=0x7f92b81a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 --2- 172.21.15.96:0/1495527701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92b8108dd0 0x7f92b81a0dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f92a800c9e0 tx=0x7f92a800cda0 comp rx=0 tx=0).stop 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bd948700 1 -- 172.21.15.96:0/1495527701 reap_dead start 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 shutdown_connections 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 >> 172.21.15.96:0/1495527701 conn(0x7f92b80ffeb0 msgr2=0x7f92b8107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 shutdown_connections 2024-04-25T17:09:02.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:02.604+0000 7f92bf3ab700 1 -- 172.21.15.96:0/1495527701 wait complete. 2024-04-25T17:09:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T17:09:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T17:09:03.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-25T17:09:03.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T17:09:03.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]: dispatch 2024-04-25T17:09:03.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-25T17:09:03.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T17:09:03.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T17:09:03.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:03 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T17:09:03.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:03 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T17:09:03.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:03 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 -- 172.21.15.96:0/3118669720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 msgr2=0x7f01281037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 --2- 172.21.15.96:0/3118669720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f01281037c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01240099c0 tx=0x7f0124009cd0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 -- 172.21.15.96:0/3118669720 shutdown_connections 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 --2- 172.21.15.96:0/3118669720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128108630 0x7f012810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 --2- 172.21.15.96:0/3118669720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f01281037c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 --2- 172.21.15.96:0/3118669720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0128107ce0 0x7f01281080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 -- 172.21.15.96:0/3118669720 >> 172.21.15.96:0/3118669720 conn(0x7f01280fee20 msgr2=0x7f0128101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 -- 172.21.15.96:0/3118669720 shutdown_connections 2024-04-25T17:09:04.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.363+0000 7f01307ef700 1 -- 172.21.15.96:0/3118669720 wait complete. 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T17:09:04.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:04 smithi096 ceph-mon[29131]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 Processor -- start 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 -- start start 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f012819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128107ce0 0x7f012819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0128108630 0x7f0128199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0128113260 con 0x7f01281033b0 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f01281130e0 con 0x7f0128107ce0 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.364+0000 7f01307ef700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01281133e0 con 0x7f0128108630 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f012819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f012819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47932/0 (socket says 172.21.15.96:47932) 2024-04-25T17:09:04.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 -- 172.21.15.96:0/3388885474 learned_addr learned my addr 172.21.15.96:0/3388885474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012dd8a700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128107ce0 0x7f012819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012ed8c700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0128108630 0x7f0128199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 -- 172.21.15.96:0/3388885474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0128108630 msgr2=0x7f0128199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0128108630 0x7f0128199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 -- 172.21.15.96:0/3388885474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128107ce0 msgr2=0x7f012819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128107ce0 0x7f012819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 -- 172.21.15.96:0/3388885474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012819a0f0 con 0x7f01281033b0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f012e58b700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f012819f820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f012400b580 tx=0x7f01240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f011f7fe700 1 -- 172.21.15.96:0/3388885474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124027070 con 0x7f01281033b0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012819a380 con 0x7f01281033b0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f011f7fe700 1 -- 172.21.15.96:0/3388885474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01240040a0 con 0x7f01281033b0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f011f7fe700 1 -- 172.21.15.96:0/3388885474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012400f460 con 0x7f01281033b0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.365+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012819a860 con 0x7f01281033b0 2024-04-25T17:09:04.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.367+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0128051190 con 0x7f01281033b0 2024-04-25T17:09:04.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.369+0000 7f011f7fe700 1 -- 172.21.15.96:0/3388885474 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f012402a070 con 0x7f01281033b0 2024-04-25T17:09:04.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.369+0000 7f011f7fe700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f01140afc10 0x7f01140b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:04.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.369+0000 7f012dd8a700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f01140afc10 0x7f01140b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:04.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.369+0000 7f011f7fe700 1 -- 172.21.15.96:0/3388885474 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f01240e0360 con 0x7f01281033b0 2024-04-25T17:09:04.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.370+0000 7f012dd8a700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f01140afc10 0x7f01140b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0128107b80 tx=0x7f011800a300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:04.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.373+0000 7f011f7fe700 1 -- 172.21.15.96:0/3388885474 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0124013070 con 0x7f01281033b0 2024-04-25T17:09:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T17:09:04.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T17:09:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:04 smithi097 ceph-mon[32602]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]': finished 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T17:09:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T17:09:04.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-25T17:09:04.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:04 smithi171 ceph-mon[33189]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T17:09:04.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.577+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f012802e510 con 0x7f01281033b0 2024-04-25T17:09:04.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.577+0000 7f011f7fe700 1 -- 172.21.15.96:0/3388885474 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f01240a8e70 con 0x7f01281033b0 2024-04-25T17:09:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:55834574923 2024-04-25T17:09:04.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f01140afc10 msgr2=0x7f01140b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f01140afc10 0x7f01140b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0128107b80 tx=0x7f011800a300 comp rx=0 tx=0).stop 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 msgr2=0x7f012819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f012819f820 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f012400b580 tx=0x7f01240047b0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 shutdown_connections 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0128108630 0x7f0128199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f01140afc10 0x7f01140b2120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01281033b0 0x7f012819f820 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 --2- 172.21.15.96:0/3388885474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0128107ce0 0x7f012819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 >> 172.21.15.96:0/3388885474 conn(0x7f01280fee20 msgr2=0x7f0128101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 shutdown_connections 2024-04-25T17:09:04.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:04.580+0000 7f01307ef700 1 -- 172.21.15.96:0/3388885474 wait complete. 2024-04-25T17:09:04.808 INFO:teuthology.orchestra.run.smithi096.stdout:304942678039 2024-04-25T17:09:04.808 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T17:09:04.819 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:05.006 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:05.006 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:05.007 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:05 smithi096 ceph-mon[29131]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T17:09:05.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:05 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3388885474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T17:09:05.469 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574923 for osd.1 2024-04-25T17:09:05.469 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:05 smithi097 ceph-mon[32602]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T17:09:05.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:05 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3388885474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T17:09:05.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:05 smithi171 ceph-mon[33189]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T17:09:05.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:05 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3388885474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T17:09:06.368 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:06 smithi096 ceph-mon[29131]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:06.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:06 smithi097 ceph-mon[32602]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:06.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:06 smithi171 ceph-mon[33189]: pgmap v309: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:08.331 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:08 smithi096 ceph-mon[29131]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T17:09:08.350 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:08.350 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:08.351 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:08.351 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:08.352 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:08.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:08 smithi097 ceph-mon[32602]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T17:09:08.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:08 smithi171 ceph-mon[33189]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 -- 172.21.15.96:0/3864688404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70103360 msgr2=0x7fce70103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 --2- 172.21.15.96:0/3864688404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70103360 0x7fce70103770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce600099c0 tx=0x7fce60009cd0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 -- 172.21.15.96:0/3864688404 shutdown_connections 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 --2- 172.21.15.96:0/3864688404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce701085e0 0x7fce7010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 --2- 172.21.15.96:0/3864688404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70103360 0x7fce70103770 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 --2- 172.21.15.96:0/3864688404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce70107c90 0x7fce701080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 -- 172.21.15.96:0/3864688404 >> 172.21.15.96:0/3864688404 conn(0x7fce700fec90 msgr2=0x7fce701010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 -- 172.21.15.96:0/3864688404 shutdown_connections 2024-04-25T17:09:09.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.178+0000 7fce7512a700 1 -- 172.21.15.96:0/3864688404 wait complete. 2024-04-25T17:09:09.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 Processor -- start 2024-04-25T17:09:09.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 -- start start 2024-04-25T17:09:09.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce70103360 0x7fce7019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70107c90 0x7fce7019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce701085e0 0x7fce70199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce70113290 con 0x7fce70107c90 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce70113110 con 0x7fce701085e0 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce7512a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fce70113410 con 0x7fce70103360 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce6e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70107c90 0x7fce7019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce6e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70107c90 0x7fce7019fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42662/0 (socket says 172.21.15.96:42662) 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce6e59c700 1 -- 172.21.15.96:0/3003452352 learned_addr learned my addr 172.21.15.96:0/3003452352 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce6ed9d700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce70103360 0x7fce7019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.179+0000 7fce6f59e700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce701085e0 0x7fce70199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce6e59c700 1 -- 172.21.15.96:0/3003452352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce70103360 msgr2=0x7fce7019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce6e59c700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce70103360 0x7fce7019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce6e59c700 1 -- 172.21.15.96:0/3003452352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce701085e0 msgr2=0x7fce70199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce6e59c700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce701085e0 0x7fce70199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce6e59c700 1 -- 172.21.15.96:0/3003452352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7019a100 con 0x7fce70107c90 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce6f59e700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce701085e0 0x7fce70199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce6e59c700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70107c90 0x7fce7019fd70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d980 tx=0x7fce6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce57fff700 1 -- 172.21.15.96:0/3003452352 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64017070 con 0x7fce70107c90 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7019a3f0 con 0x7fce70107c90 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce57fff700 1 -- 172.21.15.96:0/3003452352 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce64009de0 con 0x7fce70107c90 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce57fff700 1 -- 172.21.15.96:0/3003452352 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6400f3c0 con 0x7fce70107c90 2024-04-25T17:09:09.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.180+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a66c0 con 0x7fce70107c90 2024-04-25T17:09:09.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.181+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce70051190 con 0x7fce70107c90 2024-04-25T17:09:09.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.182+0000 7fce57fff700 1 -- 172.21.15.96:0/3003452352 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fce6400f560 con 0x7fce70107c90 2024-04-25T17:09:09.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.183+0000 7fce57fff700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fce580afc10 0x7fce580b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:09.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.183+0000 7fce6ed9d700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fce580afc10 0x7fce580b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:09.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.183+0000 7fce57fff700 1 -- 172.21.15.96:0/3003452352 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fce640de960 con 0x7fce70107c90 2024-04-25T17:09:09.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.183+0000 7fce6ed9d700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fce580afc10 0x7fce580b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fce7019ab90 tx=0x7fce6001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:09.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.188+0000 7fce57fff700 1 -- 172.21.15.96:0/3003452352 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce640a73e0 con 0x7fce70107c90 2024-04-25T17:09:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.399+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fce7019af80 con 0x7fce70107c90 2024-04-25T17:09:09.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.400+0000 7fce57fff700 1 -- 172.21.15.96:0/3003452352 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fce640a6ce0 con 0x7fce70107c90 2024-04-25T17:09:09.400 INFO:teuthology.orchestra.run.smithi096.stdout:107374182467 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fce580afc10 msgr2=0x7fce580b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fce580afc10 0x7fce580b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fce7019ab90 tx=0x7fce6001a040 comp rx=0 tx=0).stop 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70107c90 msgr2=0x7fce7019fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70107c90 0x7fce7019fd70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d980 tx=0x7fce6400dc90 comp rx=0 tx=0).stop 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 shutdown_connections 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce701085e0 0x7fce70199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fce70103360 0x7fce7019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fce580afc10 0x7fce580b2120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 --2- 172.21.15.96:0/3003452352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce70107c90 0x7fce7019fd70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 >> 172.21.15.96:0/3003452352 conn(0x7fce700fec90 msgr2=0x7fce70101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 shutdown_connections 2024-04-25T17:09:09.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:09.402+0000 7fce7512a700 1 -- 172.21.15.96:0/3003452352 wait complete. 2024-04-25T17:09:10.381 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:10 smithi096 ceph-mon[29131]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T17:09:10.381 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:10 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3003452352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T17:09:10.381 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:10 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:09:10.426 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:10 smithi097 ceph-mon[32602]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T17:09:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:10 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3003452352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T17:09:10.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:10 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:09:10.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:10 smithi171 ceph-mon[33189]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T17:09:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:10 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3003452352' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T17:09:10.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:10 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:09:11.403 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:11 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1324096305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T17:09:11.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:11 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1324096305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T17:09:11.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:11 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1324096305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T17:09:12.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 -- 172.21.15.96:0/774266896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 msgr2=0x7ff85410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/774266896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85410ff70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b0d0 tx=0x7ff84c00b3e0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 -- 172.21.15.96:0/774266896 shutdown_connections 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/774266896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85410ff70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/774266896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff854103390 0x7ff8541037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/774266896 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff854107cc0 0x7ff8541080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 -- 172.21.15.96:0/774266896 >> 172.21.15.96:0/774266896 conn(0x7ff8540fec90 msgr2=0x7ff8541010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 -- 172.21.15.96:0/774266896 shutdown_connections 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.128+0000 7ff85bbb8700 1 -- 172.21.15.96:0/774266896 wait complete. 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 Processor -- start 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 -- start start 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff854103390 0x7ff8541a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff854107cc0 0x7ff8541a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff854113260 con 0x7ff854108610 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541130e0 con 0x7ff854103390 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85bbb8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541133e0 con 0x7ff854107cc0 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42678/0 (socket says 172.21.15.96:42678) 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 -- 172.21.15.96:0/1324096305 learned_addr learned my addr 172.21.15.96:0/1324096305 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff859153700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff854107cc0 0x7ff8541a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff859954700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff854103390 0x7ff8541a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 -- 172.21.15.96:0/1324096305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff854107cc0 msgr2=0x7ff8541a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff854107cc0 0x7ff8541a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 -- 172.21.15.96:0/1324096305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff854103390 msgr2=0x7ff8541a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff854103390 0x7ff8541a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.129+0000 7ff85a155700 1 -- 172.21.15.96:0/1324096305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85419b140 con 0x7ff854108610 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.130+0000 7ff85a155700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85419aa90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009850 tx=0x7ff84c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.130+0000 7ff84affd700 1 -- 172.21.15.96:0/1324096305 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c01f070 con 0x7ff854108610 2024-04-25T17:09:12.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.130+0000 7ff84affd700 1 -- 172.21.15.96:0/1324096305 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff84c004c50 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.130+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85419b3d0 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.130+0000 7ff84affd700 1 -- 172.21.15.96:0/1324096305 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c01b690 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.130+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85419b8b0 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.131+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff854051190 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.132+0000 7ff84affd700 1 -- 172.21.15.96:0/1324096305 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ff84c022070 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.133+0000 7ff84affd700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff8400afc10 0x7ff8400b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.133+0000 7ff859954700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff8400afc10 0x7ff8400b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.133+0000 7ff84affd700 1 -- 172.21.15.96:0/1324096305 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7ff84c00f020 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.134+0000 7ff859954700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff8400afc10 0x7ff8400b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff85419bbd0 tx=0x7ff8500073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.138+0000 7ff84affd700 1 -- 172.21.15.96:0/1324096305 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff84c029020 con 0x7ff854108610 2024-04-25T17:09:12.133 INFO:teuthology.orchestra.run.smithi096.stdout:279172874269 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.368+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff85402e510 con 0x7ff854108610 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.368+0000 7ff84affd700 1 -- 172.21.15.96:0/1324096305 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff84c009cb0 con 0x7ff854108610 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff8400afc10 msgr2=0x7ff8400b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff8400afc10 0x7ff8400b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff85419bbd0 tx=0x7ff8500073d0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 msgr2=0x7ff85419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85419aa90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009850 tx=0x7ff84c007a20 comp rx=0 tx=0).stop 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 shutdown_connections 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff8400afc10 0x7ff8400b2120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff854108610 0x7ff85419aa90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff854103390 0x7ff8541a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 --2- 172.21.15.96:0/1324096305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff854107cc0 0x7ff8541a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 >> 172.21.15.96:0/1324096305 conn(0x7ff8540fec90 msgr2=0x7ff85406b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 shutdown_connections 2024-04-25T17:09:12.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:10.370+0000 7ff85bbb8700 1 -- 172.21.15.96:0/1324096305 wait complete. 2024-04-25T17:09:12.434 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:12 smithi096 ceph-mon[29131]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T17:09:12.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:12 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:12 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:12 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.435 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:12 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:12 smithi097 ceph-mon[32602]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T17:09:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:12 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:12 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:12 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:12 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:12 smithi171 ceph-mon[33189]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T17:09:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:12 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:12 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:12 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:12 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:12.745 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874267 got 279172874269 for osd.10 2024-04-25T17:09:12.746 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:13.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:13 smithi096 ceph-mon[29131]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 927 B/s rd, 1 op/s; 58 B/s, 0 objects/s recovering 2024-04-25T17:09:13.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:13 smithi097 ceph-mon[32602]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 927 B/s rd, 1 op/s; 58 B/s, 0 objects/s recovering 2024-04-25T17:09:13.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:13 smithi171 ceph-mon[33189]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 927 B/s rd, 1 op/s; 58 B/s, 0 objects/s recovering 2024-04-25T17:09:14.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 -- 172.21.15.96:0/2500325467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 msgr2=0x7f931c10ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:14.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 --2- 172.21.15.96:0/2500325467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c10ff40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f931400b0d0 tx=0x7f931400b3e0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 -- 172.21.15.96:0/2500325467 shutdown_connections 2024-04-25T17:09:14.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 --2- 172.21.15.96:0/2500325467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c10ff40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 --2- 172.21.15.96:0/2500325467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f931c103360 0x7f931c103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 --2- 172.21.15.96:0/2500325467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f931c107c90 0x7f931c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 -- 172.21.15.96:0/2500325467 >> 172.21.15.96:0/2500325467 conn(0x7f931c0fecb0 msgr2=0x7f931c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 -- 172.21.15.96:0/2500325467 shutdown_connections 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.286+0000 7f9321de3700 1 -- 172.21.15.96:0/2500325467 wait complete. 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 Processor -- start 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 -- start start 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f931c103360 0x7f931c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f931c107c90 0x7f931c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f931c114380 con 0x7f931c1085e0 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f931c114200 con 0x7f931c103360 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f9321de3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f931c114500 con 0x7f931c107c90 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f931bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f931bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42692/0 (socket says 172.21.15.96:42692) 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.287+0000 7f931bfff700 1 -- 172.21.15.96:0/1612549956 learned_addr learned my addr 172.21.15.96:0/1612549956 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f931b7fe700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f931c103360 0x7f931c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f9312ffd700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f931c107c90 0x7f931c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:14.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f931bfff700 1 -- 172.21.15.96:0/1612549956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f931c107c90 msgr2=0x7f931c1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f931bfff700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f931c107c90 0x7f931c1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f931bfff700 1 -- 172.21.15.96:0/1612549956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f931c103360 msgr2=0x7f931c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f931bfff700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f931c103360 0x7f931c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f931bfff700 1 -- 172.21.15.96:0/1612549956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c19b0e0 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f931bfff700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c19aaa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9314009fd0 tx=0x7f9314007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f93197fa700 1 -- 172.21.15.96:0/1612549956 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931401f070 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f93197fa700 1 -- 172.21.15.96:0/1612549956 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9314003c00 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f93197fa700 1 -- 172.21.15.96:0/1612549956 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931401b7b0 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c19b370 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.288+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c19b800 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.289+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931c051190 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.291+0000 7f93197fa700 1 -- 172.21.15.96:0/1612549956 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9314022070 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.291+0000 7f93197fa700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f93080afc10 0x7f93080b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.291+0000 7f931b7fe700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f93080afc10 0x7f93080b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.291+0000 7f93197fa700 1 -- 172.21.15.96:0/1612549956 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f931400f020 con 0x7f931c1085e0 2024-04-25T17:09:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.292+0000 7f931b7fe700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f93080afc10 0x7f93080b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f931c19bbe0 tx=0x7f930c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:14.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.295+0000 7f93197fa700 1 -- 172.21.15.96:0/1612549956 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9314029020 con 0x7f931c1085e0 2024-04-25T17:09:14.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.503+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f931c02e510 con 0x7f931c1085e0 2024-04-25T17:09:14.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.504+0000 7f93197fa700 1 -- 172.21.15.96:0/1612549956 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f93140a8fb0 con 0x7f931c1085e0 2024-04-25T17:09:14.503 INFO:teuthology.orchestra.run.smithi096.stdout:231928234025 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f93080afc10 msgr2=0x7f93080b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f93080afc10 0x7f93080b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f931c19bbe0 tx=0x7f930c009190 comp rx=0 tx=0).stop 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 msgr2=0x7f931c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c19aaa0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9314009fd0 tx=0x7f9314007a20 comp rx=0 tx=0).stop 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 shutdown_connections 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f93080afc10 0x7f93080b2120 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f931c1085e0 0x7f931c19aaa0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f931c103360 0x7f931c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 --2- 172.21.15.96:0/1612549956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f931c107c90 0x7f931c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:14.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 >> 172.21.15.96:0/1612549956 conn(0x7f931c0fecb0 msgr2=0x7f931c107370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:14.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 shutdown_connections 2024-04-25T17:09:14.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.506+0000 7f9321de3700 1 -- 172.21.15.96:0/1612549956 wait complete. 2024-04-25T17:09:14.895 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:14 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1612549956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T17:09:14.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:14 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1612549956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T17:09:15.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:14 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1612549956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T17:09:15.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:15 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1415691519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T17:09:15.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:15 smithi096 ceph-mon[29131]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 53 B/s, 0 objects/s recovering 2024-04-25T17:09:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:15 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1415691519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T17:09:15.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:15 smithi097 ceph-mon[32602]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 53 B/s, 0 objects/s recovering 2024-04-25T17:09:16.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:15 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1415691519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T17:09:16.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:15 smithi171 ceph-mon[33189]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s; 53 B/s, 0 objects/s recovering 2024-04-25T17:09:17.127 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182467 for osd.3 2024-04-25T17:09:17.127 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:17.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.680+0000 7fd7293f1700 1 -- 172.21.15.96:0/4038398578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724108610 msgr2=0x7fd72410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:17.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.680+0000 7fd7293f1700 1 --2- 172.21.15.96:0/4038398578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724108610 0x7fd72410ff70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b0d0 tx=0x7fd71c00b3e0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 -- 172.21.15.96:0/4038398578 shutdown_connections 2024-04-25T17:09:17.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 --2- 172.21.15.96:0/4038398578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724108610 0x7fd72410ff70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 --2- 172.21.15.96:0/4038398578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724103390 0x7fd7241037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 --2- 172.21.15.96:0/4038398578 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724107cc0 0x7fd7241080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 -- 172.21.15.96:0/4038398578 >> 172.21.15.96:0/4038398578 conn(0x7fd7240fec90 msgr2=0x7fd7241010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 -- 172.21.15.96:0/4038398578 shutdown_connections 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 -- 172.21.15.96:0/4038398578 wait complete. 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 Processor -- start 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.681+0000 7fd7293f1700 1 -- start start 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd7293f1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724103390 0x7fd72419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd7293f1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724107cc0 0x7fd72419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd7293f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724108610 0x7fd72419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd7293f1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd724113240 con 0x7fd724103390 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd7293f1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241130c0 con 0x7fd724108610 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd7293f1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7241133c0 con 0x7fd724107cc0 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724103390 0x7fd72419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724103390 0x7fd72419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42712/0 (socket says 172.21.15.96:42712) 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 -- 172.21.15.96:0/1415691519 learned_addr learned my addr 172.21.15.96:0/1415691519 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd71a7fc700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724107cc0 0x7fd72419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:17.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd7237fe700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724108610 0x7fd72419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:17.592 INFO:teuthology.orchestra.run.smithi096.stdout:77309411401 2024-04-25T17:09:17.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 -- 172.21.15.96:0/1415691519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724107cc0 msgr2=0x7fd72419c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:17.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724107cc0 0x7fd72419c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 -- 172.21.15.96:0/1415691519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724108610 msgr2=0x7fd72419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:17.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724108610 0x7fd72419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 -- 172.21.15.96:0/1415691519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7241a6ed0 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd722ffd700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724103390 0x7fd72419bb80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d950 tx=0x7fd71400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.682+0000 7fd720ff9700 1 -- 172.21.15.96:0/1415691519 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714016070 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.683+0000 7fd720ff9700 1 -- 172.21.15.96:0/1415691519 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd714009df0 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.683+0000 7fd720ff9700 1 -- 172.21.15.96:0/1415691519 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71400f3c0 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.683+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7241a71c0 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.683+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7241a7700 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.683+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd724051190 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.685+0000 7fd720ff9700 1 -- 172.21.15.96:0/1415691519 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fd714019070 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.685+0000 7fd720ff9700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd7100afc90 0x7fd7100b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.686+0000 7fd71a7fc700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd7100afc90 0x7fd7100b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.686+0000 7fd720ff9700 1 -- 172.21.15.96:0/1415691519 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fd7140dea20 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.686+0000 7fd71a7fc700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd7100afc90 0x7fd7100b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd724106590 tx=0x7fd70c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.689+0000 7fd720ff9700 1 -- 172.21.15.96:0/1415691519 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7140a7460 con 0x7fd724103390 2024-04-25T17:09:17.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.895+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd72402e510 con 0x7fd724103390 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.896+0000 7fd720ff9700 1 -- 172.21.15.96:0/1415691519 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd7140a6d60 con 0x7fd724103390 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd7100afc90 msgr2=0x7fd7100b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd7100afc90 0x7fd7100b21a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd724106590 tx=0x7fd70c00b410 comp rx=0 tx=0).stop 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724103390 msgr2=0x7fd72419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724103390 0x7fd72419bb80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd71400d950 tx=0x7fd71400dc60 comp rx=0 tx=0).stop 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 shutdown_connections 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd724108610 0x7fd72419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fd7100afc90 0x7fd7100b21a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd724103390 0x7fd72419bb80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 --2- 172.21.15.96:0/1415691519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724107cc0 0x7fd72419c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 >> 172.21.15.96:0/1415691519 conn(0x7fd7240fec90 msgr2=0x7fd724107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 shutdown_connections 2024-04-25T17:09:17.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:14.898+0000 7fd7293f1700 1 -- 172.21.15.96:0/1415691519 wait complete. 2024-04-25T17:09:18.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411401 for osd.2 2024-04-25T17:09:18.144 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:18.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:18 smithi096 ceph-mon[29131]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T17:09:18.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:18 smithi097 ceph-mon[32602]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T17:09:18.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:18 smithi171 ceph-mon[33189]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T17:09:18.877 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234020 got 231928234025 for osd.8 2024-04-25T17:09:18.878 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:20.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:20 smithi096 ceph-mon[29131]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T17:09:20.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:20 smithi097 ceph-mon[32602]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T17:09:20.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:20 smithi171 ceph-mon[33189]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 -- 172.21.15.96:0/3955367382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 msgr2=0x7f9628108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 --2- 172.21.15.96:0/3955367382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f9628108cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f961800aaa0 tx=0x7f961800adb0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 -- 172.21.15.96:0/3955367382 shutdown_connections 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 --2- 172.21.15.96:0/3955367382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96281031e0 0x7f962810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 --2- 172.21.15.96:0/3955367382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f9628108cd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 --2- 172.21.15.96:0/3955367382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9628102890 0x7f9628102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 -- 172.21.15.96:0/3955367382 >> 172.21.15.96:0/3955367382 conn(0x7f96280fe5c0 msgr2=0x7f96281009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.596+0000 7f962d034700 1 -- 172.21.15.96:0/3955367382 shutdown_connections 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.597+0000 7f962d034700 1 -- 172.21.15.96:0/3955367382 wait complete. 2024-04-25T17:09:20.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 Processor -- start 2024-04-25T17:09:20.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 -- start start 2024-04-25T17:09:20.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9628102890 0x7f96281a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:20.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96281031e0 0x7f96281a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:20.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f96281a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9628112210 con 0x7f96281088c0 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9628112090 con 0x7f96281031e0 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.598+0000 7f962d034700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9628112390 con 0x7f9628102890 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f96281a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f96281a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59608/0 (socket says 172.21.15.96:59608) 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 -- 172.21.15.96:0/3788540253 learned_addr learned my addr 172.21.15.96:0/3788540253 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962659c700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96281031e0 0x7f96281a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f9626d9d700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9628102890 0x7f96281a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 -- 172.21.15.96:0/3788540253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9628102890 msgr2=0x7f96281a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9628102890 0x7f96281a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 -- 172.21.15.96:0/3788540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96281031e0 msgr2=0x7f96281a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96281031e0 0x7f96281a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 -- 172.21.15.96:0/3788540253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96281a6170 con 0x7f96281088c0 2024-04-25T17:09:20.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f962759e700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f96281a5b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9620007ee0 tx=0x7f962000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:20.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.599+0000 7f960ffff700 1 -- 172.21.15.96:0/3788540253 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620004bb0 con 0x7f96281088c0 2024-04-25T17:09:20.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.600+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96281a6460 con 0x7f96281088c0 2024-04-25T17:09:20.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.600+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281a69a0 con 0x7f96281088c0 2024-04-25T17:09:20.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.600+0000 7f960ffff700 1 -- 172.21.15.96:0/3788540253 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9620004d50 con 0x7f96281088c0 2024-04-25T17:09:20.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.600+0000 7f960ffff700 1 -- 172.21.15.96:0/3788540253 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620005920 con 0x7f96281088c0 2024-04-25T17:09:20.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.601+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9614005330 con 0x7f96281088c0 2024-04-25T17:09:20.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.602+0000 7f960ffff700 1 -- 172.21.15.96:0/3788540253 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f962001a020 con 0x7f96281088c0 2024-04-25T17:09:20.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.603+0000 7f960ffff700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f96100afc90 0x7f96100b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:20.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.603+0000 7f9626d9d700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f96100afc90 0x7f96100b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:20.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.603+0000 7f960ffff700 1 -- 172.21.15.96:0/3788540253 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f96200de8a0 con 0x7f96281088c0 2024-04-25T17:09:20.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.604+0000 7f9626d9d700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f96100afc90 0x7f96100b21a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f961800a860 tx=0x7f961800c650 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:20.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.607+0000 7f960ffff700 1 -- 172.21.15.96:0/3788540253 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9620010070 con 0x7f96281088c0 2024-04-25T17:09:20.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.808+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f96140058f0 con 0x7f96281088c0 2024-04-25T17:09:20.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.809+0000 7f960ffff700 1 -- 172.21.15.96:0/3788540253 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f96200a7320 con 0x7f96281088c0 2024-04-25T17:09:20.807 INFO:teuthology.orchestra.run.smithi096.stdout:133143986240 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f96100afc90 msgr2=0x7f96100b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f96100afc90 0x7f96100b21a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f961800a860 tx=0x7f961800c650 comp rx=0 tx=0).stop 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 msgr2=0x7f96281a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f96281a5b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9620007ee0 tx=0x7f962000d3b0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 shutdown_connections 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f96100afc90 0x7f96100b21a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96281088c0 0x7f96281a5b80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9628102890 0x7f96281a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 --2- 172.21.15.96:0/3788540253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96281031e0 0x7f96281a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 >> 172.21.15.96:0/3788540253 conn(0x7f96280fe5c0 msgr2=0x7f9628100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 shutdown_connections 2024-04-25T17:09:20.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:20.811+0000 7f962d034700 1 -- 172.21.15.96:0/3788540253 wait complete. 2024-04-25T17:09:21.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:21 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3788540253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T17:09:21.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:21 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3788540253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T17:09:21.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:21 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3788540253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T17:09:22.339 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:22 smithi096 ceph-mon[29131]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:22.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:22 smithi097 ceph-mon[32602]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:22.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:22 smithi171 ceph-mon[33189]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:22.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 -- 172.21.15.96:0/3297703919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc108db0 msgr2=0x7fdacc1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:22.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 --2- 172.21.15.96:0/3297703919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc108db0 0x7fdacc1091c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdac00099c0 tx=0x7fdac0009cd0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 -- 172.21.15.96:0/3297703919 shutdown_connections 2024-04-25T17:09:22.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 --2- 172.21.15.96:0/3297703919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdacc109700 0x7fdacc111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 --2- 172.21.15.96:0/3297703919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc104480 0x7fdacc104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 --2- 172.21.15.96:0/3297703919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc108db0 0x7fdacc1091c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 -- 172.21.15.96:0/3297703919 >> 172.21.15.96:0/3297703919 conn(0x7fdacc0ffd70 msgr2=0x7fdacc1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:22.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 -- 172.21.15.96:0/3297703919 shutdown_connections 2024-04-25T17:09:22.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 -- 172.21.15.96:0/3297703919 wait complete. 2024-04-25T17:09:22.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 Processor -- start 2024-04-25T17:09:22.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.771+0000 7fdad23bb700 1 -- start start 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad23bb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdacc104480 0x7fdacc19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad23bb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc108db0 0x7fdacc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad23bb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc109700 0x7fdacc1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad23bb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc114310 con 0x7fdacc109700 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad23bb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc114190 con 0x7fdacc108db0 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad23bb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc114490 con 0x7fdacc104480 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc109700 0x7fdacc1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc109700 0x7fdacc1a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59628/0 (socket says 172.21.15.96:59628) 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 -- 172.21.15.96:0/2523242134 learned_addr learned my addr 172.21.15.96:0/2523242134 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdacbfff700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdacc104480 0x7fdacc19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdacb7fe700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc108db0 0x7fdacc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 -- 172.21.15.96:0/2523242134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdacc104480 msgr2=0x7fdacc19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:22.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdacc104480 0x7fdacc19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 -- 172.21.15.96:0/2523242134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc108db0 msgr2=0x7fdacc19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc108db0 0x7fdacc19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.772+0000 7fdad0958700 1 -- 172.21.15.96:0/2523242134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc1a1cc0 con 0x7fdacc109700 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.773+0000 7fdad0958700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc109700 0x7fdacc1a16d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007ee0 tx=0x7fdac400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.773+0000 7fdac97fa700 1 -- 172.21.15.96:0/2523242134 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac400dbd0 con 0x7fdacc109700 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.773+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc1a1fb0 con 0x7fdacc109700 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.773+0000 7fdac97fa700 1 -- 172.21.15.96:0/2523242134 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdac400dd70 con 0x7fdacc109700 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.773+0000 7fdac97fa700 1 -- 172.21.15.96:0/2523242134 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4005990 con 0x7fdacc109700 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.773+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc1a2470 con 0x7fdacc109700 2024-04-25T17:09:22.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.774+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc051190 con 0x7fdacc109700 2024-04-25T17:09:22.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.775+0000 7fdac97fa700 1 -- 172.21.15.96:0/2523242134 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fdac401a020 con 0x7fdacc109700 2024-04-25T17:09:22.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.776+0000 7fdac97fa700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdab40afc10 0x7fdab40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:22.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.776+0000 7fdacbfff700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdab40afc10 0x7fdab40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:22.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.776+0000 7fdac97fa700 1 -- 172.21.15.96:0/2523242134 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fdac40de530 con 0x7fdacc109700 2024-04-25T17:09:22.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.776+0000 7fdacbfff700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdab40afc10 0x7fdab40b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ac50 tx=0x7fdabc00a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:22.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.781+0000 7fdac97fa700 1 -- 172.21.15.96:0/2523242134 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdac4010070 con 0x7fdacc109700 2024-04-25T17:09:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.987+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdacc1a9980 con 0x7fdacc109700 2024-04-25T17:09:22.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.987+0000 7fdac97fa700 1 -- 172.21.15.96:0/2523242134 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdac40a6fb0 con 0x7fdacc109700 2024-04-25T17:09:22.986 INFO:teuthology.orchestra.run.smithi096.stdout:34359738452 2024-04-25T17:09:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdab40afc10 msgr2=0x7fdab40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdab40afc10 0x7fdab40b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ac50 tx=0x7fdabc00a380 comp rx=0 tx=0).stop 2024-04-25T17:09:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc109700 msgr2=0x7fdacc1a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc109700 0x7fdacc1a16d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007ee0 tx=0x7fdac400d3b0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 shutdown_connections 2024-04-25T17:09:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdab40afc10 0x7fdab40b2120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdacc109700 0x7fdacc1a16d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdacc104480 0x7fdacc19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 --2- 172.21.15.96:0/2523242134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdacc108db0 0x7fdacc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 >> 172.21.15.96:0/2523242134 conn(0x7fdacc0ffd70 msgr2=0x7fdacc110d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 shutdown_connections 2024-04-25T17:09:22.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:22.989+0000 7fdad23bb700 1 -- 172.21.15.96:0/2523242134 wait complete. 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 -- 172.21.15.96:0/1945134432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa8108df0 msgr2=0x7f8fa8109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/1945134432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa8108df0 0x7f8fa8109200 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0099c0 tx=0x7f8f9c009cd0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 -- 172.21.15.96:0/1945134432 shutdown_connections 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/1945134432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8109740 0x7f8fa81110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/1945134432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fa81044c0 0x7f8fa81048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/1945134432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa8108df0 0x7f8fa8109200 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 -- 172.21.15.96:0/1945134432 >> 172.21.15.96:0/1945134432 conn(0x7f8fa80ffeb0 msgr2=0x7f8fa81022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:23.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 -- 172.21.15.96:0/1945134432 shutdown_connections 2024-04-25T17:09:23.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.069+0000 7f8fae4cf700 1 -- 172.21.15.96:0/1945134432 wait complete. 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 Processor -- start 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 -- start start 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa81044c0 0x7f8fa819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fa8108df0 0x7f8fa819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8109740 0x7f8fa819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81143c0 con 0x7f8fa81044c0 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8114240 con 0x7f8fa8109740 2024-04-25T17:09:23.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fae4cf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8114540 con 0x7f8fa8108df0 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fa7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa81044c0 0x7f8fa819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fa7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa81044c0 0x7f8fa819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59636/0 (socket says 172.21.15.96:59636) 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fa7fff700 1 -- 172.21.15.96:0/3635265279 learned_addr learned my addr 172.21.15.96:0/3635265279 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8fa77fe700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fa8108df0 0x7f8fa819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.070+0000 7f8faca6c700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8109740 0x7f8fa819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa7fff700 1 -- 172.21.15.96:0/3635265279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fa8108df0 msgr2=0x7f8fa819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa7fff700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fa8108df0 0x7f8fa819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa7fff700 1 -- 172.21.15.96:0/3635265279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8109740 msgr2=0x7f8fa819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa7fff700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8109740 0x7f8fa819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa7fff700 1 -- 172.21.15.96:0/3635265279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa819fc80 con 0x7f8fa81044c0 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa7fff700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa81044c0 0x7f8fa819adb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800b530 tx=0x7f8f9800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa57fa700 1 -- 172.21.15.96:0/3635265279 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98015020 con 0x7f8fa81044c0 2024-04-25T17:09:23.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa819ff70 con 0x7f8fa81044c0 2024-04-25T17:09:23.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa57fa700 1 -- 172.21.15.96:0/3635265279 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f98011da0 con 0x7f8fa81044c0 2024-04-25T17:09:23.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fa57fa700 1 -- 172.21.15.96:0/3635265279 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800f2e0 con 0x7f8fa81044c0 2024-04-25T17:09:23.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.071+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa81a7890 con 0x7f8fa81044c0 2024-04-25T17:09:23.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.072+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa8051190 con 0x7f8fa81044c0 2024-04-25T17:09:23.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.074+0000 7f8fa57fa700 1 -- 172.21.15.96:0/3635265279 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8f98018020 con 0x7f8fa81044c0 2024-04-25T17:09:23.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.074+0000 7f8fa57fa700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8f900afc10 0x7f8f900b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.074+0000 7f8fa77fe700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8f900afc10 0x7f8f900b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:23.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.074+0000 7f8fa57fa700 1 -- 172.21.15.96:0/3635265279 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f8f980de530 con 0x7f8fa81044c0 2024-04-25T17:09:23.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.075+0000 7f8fa77fe700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8f900afc10 0x7f8f900b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c006010 tx=0x7f8f9c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:23.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.079+0000 7f8fa57fa700 1 -- 172.21.15.96:0/3635265279 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f9801f020 con 0x7f8fa81044c0 2024-04-25T17:09:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.283+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8fa819c110 con 0x7f8fa81044c0 2024-04-25T17:09:23.283 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:23 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2523242134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T17:09:23.283 INFO:teuthology.orchestra.run.smithi096.stdout:180388626486 2024-04-25T17:09:23.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.284+0000 7f8fa57fa700 1 -- 172.21.15.96:0/3635265279 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8f980a70c0 con 0x7f8fa81044c0 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.286+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8f900afc10 msgr2=0x7f8f900b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.286+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8f900afc10 0x7f8f900b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c006010 tx=0x7f8f9c000bc0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.286+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa81044c0 msgr2=0x7f8fa819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.286+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa81044c0 0x7f8fa819adb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800b530 tx=0x7f8f9800b840 comp rx=0 tx=0).stop 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 shutdown_connections 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8109740 0x7f8fa819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f8f900afc10 0x7f8f900b2120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fa81044c0 0x7f8fa819adb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 --2- 172.21.15.96:0/3635265279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fa8108df0 0x7f8fa819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 >> 172.21.15.96:0/3635265279 conn(0x7f8fa80ffeb0 msgr2=0x7f8fa8106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 shutdown_connections 2024-04-25T17:09:23.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.287+0000 7f8fae4cf700 1 -- 172.21.15.96:0/3635265279 wait complete. 2024-04-25T17:09:23.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:23 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2523242134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T17:09:23.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:23 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2523242134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T17:09:23.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 -- 172.21.15.96:0/807851340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068103390 msgr2=0x7f90681037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 --2- 172.21.15.96:0/807851340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068103390 0x7f90681037a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f90580099c0 tx=0x7f9058009cd0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 -- 172.21.15.96:0/807851340 shutdown_connections 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 --2- 172.21.15.96:0/807851340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9068108610 0x7f906810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 --2- 172.21.15.96:0/807851340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068103390 0x7f90681037a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 --2- 172.21.15.96:0/807851340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9068107cc0 0x7f90681080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 -- 172.21.15.96:0/807851340 >> 172.21.15.96:0/807851340 conn(0x7f90680fec90 msgr2=0x7f90681010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 -- 172.21.15.96:0/807851340 shutdown_connections 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 -- 172.21.15.96:0/807851340 wait complete. 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 Processor -- start 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.914+0000 7f906e48b700 1 -- start start 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f906e48b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9068103390 0x7f90681a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f906e48b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068107cc0 0x7f90681a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f906e48b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9068108610 0x7f906819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f906e48b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9068113240 con 0x7f9068107cc0 2024-04-25T17:09:23.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f906e48b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90681130c0 con 0x7f9068108610 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f906e48b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90681133c0 con 0x7f9068103390 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068107cc0 0x7f90681a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068107cc0 0x7f90681a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59652/0 (socket says 172.21.15.96:59652) 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 -- 172.21.15.96:0/1101138139 learned_addr learned my addr 172.21.15.96:0/1101138139 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f9067fff700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9068103390 0x7f90681a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 -- 172.21.15.96:0/1101138139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9068103390 msgr2=0x7f90681a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9068103390 0x7f90681a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 -- 172.21.15.96:0/1101138139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9068108610 msgr2=0x7f906819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9068108610 0x7f906819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 -- 172.21.15.96:0/1101138139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906819b1d0 con 0x7f9068107cc0 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90677fe700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068107cc0 0x7f90681a0e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f905c00d980 tx=0x7f905c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.915+0000 7f90657fa700 1 -- 172.21.15.96:0/1101138139 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c017070 con 0x7f9068107cc0 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.916+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906819b4c0 con 0x7f9068107cc0 2024-04-25T17:09:23.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.916+0000 7f90657fa700 1 -- 172.21.15.96:0/1101138139 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c010460 con 0x7f9068107cc0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.916+0000 7f90657fa700 1 -- 172.21.15.96:0/1101138139 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c00de60 con 0x7f9068107cc0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.916+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a7760 con 0x7f9068107cc0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.917+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9068051190 con 0x7f9068107cc0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.918+0000 7f90657fa700 1 -- 172.21.15.96:0/1101138139 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f905c010610 con 0x7f9068107cc0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.919+0000 7f90657fa700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90500afc10 0x7f90500b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.919+0000 7f9067fff700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90500afc10 0x7f90500b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.919+0000 7f90657fa700 1 -- 172.21.15.96:0/1101138139 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f905c0de960 con 0x7f9068107cc0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.919+0000 7f9067fff700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90500afc10 0x7f90500b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9058003800 tx=0x7f90580095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:23.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:23.922+0000 7f90657fa700 1 -- 172.21.15.96:0/1101138139 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f905c0a73e0 con 0x7f9068107cc0 2024-04-25T17:09:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.123+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f90681a7930 con 0x7f9068107cc0 2024-04-25T17:09:24.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.123+0000 7f90657fa700 1 -- 172.21.15.96:0/1101138139 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f905c0a6ce0 con 0x7f9068107cc0 2024-04-25T17:09:24.122 INFO:teuthology.orchestra.run.smithi096.stdout:206158430257 2024-04-25T17:09:24.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90500afc10 msgr2=0x7f90500b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:24.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90500afc10 0x7f90500b2120 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9058003800 tx=0x7f90580095f0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068107cc0 msgr2=0x7f90681a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068107cc0 0x7f90681a0e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f905c00d980 tx=0x7f905c00dd40 comp rx=0 tx=0).stop 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 shutdown_connections 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9068108610 0x7f906819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9068103390 0x7f90681a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f90500afc10 0x7f90500b2120 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 --2- 172.21.15.96:0/1101138139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9068107cc0 0x7f90681a0e10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.126+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 >> 172.21.15.96:0/1101138139 conn(0x7f90680fec90 msgr2=0x7f9068109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.127+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 shutdown_connections 2024-04-25T17:09:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.127+0000 7f906e48b700 1 -- 172.21.15.96:0/1101138139 wait complete. 2024-04-25T17:09:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:24 smithi096 ceph-mon[29131]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:24 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3635265279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T17:09:24.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:24 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1101138139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T17:09:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:24 smithi097 ceph-mon[32602]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:24 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3635265279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T17:09:24.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:24 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1101138139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T17:09:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:24 smithi171 ceph-mon[33189]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:24 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3635265279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T17:09:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:24 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1101138139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T17:09:24.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626482 got 180388626486 for osd.6 2024-04-25T17:09:24.551 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:24.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 -- 172.21.15.96:0/987252468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 msgr2=0x7f155c103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:24.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 --2- 172.21.15.96:0/987252468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c103500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f154c009990 tx=0x7f154c009ca0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 -- 172.21.15.96:0/987252468 shutdown_connections 2024-04-25T17:09:24.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 --2- 172.21.15.96:0/987252468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f155c108540 0x7f155c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 --2- 172.21.15.96:0/987252468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c103500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 --2- 172.21.15.96:0/987252468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c103a40 0x7f155c107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 -- 172.21.15.96:0/987252468 >> 172.21.15.96:0/987252468 conn(0x7f155c0fec90 msgr2=0x7f155c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:24.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 -- 172.21.15.96:0/987252468 shutdown_connections 2024-04-25T17:09:24.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.855+0000 7f1562159700 1 -- 172.21.15.96:0/987252468 wait complete. 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 Processor -- start 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 -- start start 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f155c103a40 0x7f155c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c108540 0x7f155c1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f155c11a8e0 con 0x7f155c1030f0 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f155c11a760 con 0x7f155c103a40 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f1562159700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f155c11aa60 con 0x7f155c108540 2024-04-25T17:09:24.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.856+0000 7f155b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c1a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59674/0 (socket says 172.21.15.96:59674) 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155b7fe700 1 -- 172.21.15.96:0/1797233141 learned_addr learned my addr 172.21.15.96:0/1797233141 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155affd700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f155c103a40 0x7f155c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155bfff700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c108540 0x7f155c1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155affd700 1 -- 172.21.15.96:0/1797233141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c108540 msgr2=0x7f155c1a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155affd700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c108540 0x7f155c1a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155affd700 1 -- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 msgr2=0x7f155c1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155affd700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c1a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155affd700 1 -- 172.21.15.96:0/1797233141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c1a6100 con 0x7f155c103a40 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155b7fe700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155bfff700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c108540 0x7f155c1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.857+0000 7f155affd700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f155c103a40 0x7f155c1a17d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1554007ec0 tx=0x7f155400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:24.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.858+0000 7f1558ff9700 1 -- 172.21.15.96:0/1797233141 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400ddd0 con 0x7f155c103a40 2024-04-25T17:09:24.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.858+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c1a63f0 con 0x7f155c103a40 2024-04-25T17:09:24.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.858+0000 7f1558ff9700 1 -- 172.21.15.96:0/1797233141 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1554004bb0 con 0x7f155c103a40 2024-04-25T17:09:24.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.858+0000 7f1558ff9700 1 -- 172.21.15.96:0/1797233141 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554015400 con 0x7f155c103a40 2024-04-25T17:09:24.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.858+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c1add10 con 0x7f155c103a40 2024-04-25T17:09:24.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.860+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155c051190 con 0x7f155c103a40 2024-04-25T17:09:24.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.865+0000 7f1558ff9700 1 -- 172.21.15.96:0/1797233141 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1554005000 con 0x7f155c103a40 2024-04-25T17:09:24.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.866+0000 7f1558ff9700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f15440afd10 0x7f15440b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:24.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.866+0000 7f155b7fe700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f15440afd10 0x7f15440b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:24.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.866+0000 7f1558ff9700 1 -- 172.21.15.96:0/1797233141 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f15540df160 con 0x7f155c103a40 2024-04-25T17:09:24.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.866+0000 7f155b7fe700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f15440afd10 0x7f15440b2220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f154c003800 tx=0x7f154c01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:24.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:24.868+0000 7f1558ff9700 1 -- 172.21.15.96:0/1797233141 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1554010070 con 0x7f155c103a40 2024-04-25T17:09:25.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.071+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f155c1adee0 con 0x7f155c103a40 2024-04-25T17:09:25.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.071+0000 7f1558ff9700 1 -- 172.21.15.96:0/1797233141 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f15540a7ba0 con 0x7f155c103a40 2024-04-25T17:09:25.070 INFO:teuthology.orchestra.run.smithi096.stdout:257698037798 2024-04-25T17:09:25.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.073+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f15440afd10 msgr2=0x7f15440b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.073+0000 7f1562159700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f15440afd10 0x7f15440b2220 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f154c003800 tx=0x7f154c01a040 comp rx=0 tx=0).stop 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.073+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f155c103a40 msgr2=0x7f155c1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.073+0000 7f1562159700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f155c103a40 0x7f155c1a17d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1554007ec0 tx=0x7f155400d3b0 comp rx=0 tx=0).stop 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 shutdown_connections 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f155c108540 0x7f155c1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f15440afd10 0x7f15440b2220 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f155c1030f0 0x7f155c1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 --2- 172.21.15.96:0/1797233141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f155c103a40 0x7f155c1a17d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 >> 172.21.15.96:0/1797233141 conn(0x7f155c0fec90 msgr2=0x7f155c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 shutdown_connections 2024-04-25T17:09:25.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:25.074+0000 7f1562159700 1 -- 172.21.15.96:0/1797233141 wait complete. 2024-04-25T17:09:25.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986234 got 133143986240 for osd.4 2024-04-25T17:09:25.280 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:25.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:25 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1797233141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T17:09:25.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:25 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1797233141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T17:09:25.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:25 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1797233141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T17:09:26.329 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738452 for osd.0 2024-04-25T17:09:26.329 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:26.429 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:26 smithi096 ceph-mon[29131]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:26.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:26 smithi097 ceph-mon[32602]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:26.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:26 smithi171 ceph-mon[33189]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:26.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 -- 172.21.15.96:0/2768228616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc108890 msgr2=0x7fcdbc108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:26.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 --2- 172.21.15.96:0/2768228616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc108890 0x7fcdbc108ca0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009990 tx=0x7fcdac009ca0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 -- 172.21.15.96:0/2768228616 shutdown_connections 2024-04-25T17:09:26.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 --2- 172.21.15.96:0/2768228616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1031b0 0x7fcdbc10f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 --2- 172.21.15.96:0/2768228616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc108890 0x7fcdbc108ca0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 --2- 172.21.15.96:0/2768228616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdbc102860 0x7fcdbc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 -- 172.21.15.96:0/2768228616 >> 172.21.15.96:0/2768228616 conn(0x7fcdbc0fe3f0 msgr2=0x7fcdbc100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:26.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 -- 172.21.15.96:0/2768228616 shutdown_connections 2024-04-25T17:09:26.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.776+0000 7fcdc108c700 1 -- 172.21.15.96:0/2768228616 wait complete. 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 Processor -- start 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 -- start start 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdbc102860 0x7fcdbc19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc1031b0 0x7fcdbc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc108890 0x7fcdbc19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc112a10 con 0x7fcdbc1031b0 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc112890 con 0x7fcdbc108890 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.777+0000 7fcdc108c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc112b90 con 0x7fcdbc102860 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc1031b0 0x7fcdbc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc1031b0 0x7fcdbc19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59682/0 (socket says 172.21.15.96:59682) 2024-04-25T17:09:26.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 -- 172.21.15.96:0/4226486367 learned_addr learned my addr 172.21.15.96:0/4226486367 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdbb59e700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc108890 0x7fcdbc19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdbad9d700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdbc102860 0x7fcdbc19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 -- 172.21.15.96:0/4226486367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdbc102860 msgr2=0x7fcdbc19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdbc102860 0x7fcdbc19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 -- 172.21.15.96:0/4226486367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc108890 msgr2=0x7fcdbc19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc108890 0x7fcdbc19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 -- 172.21.15.96:0/4226486367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc19fc20 con 0x7fcdbc1031b0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdbb59e700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc108890 0x7fcdbc19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdba59c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc1031b0 0x7fcdbc19b2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b500 tx=0x7fcdb000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcda3fff700 1 -- 172.21.15.96:0/4226486367 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0015020 con 0x7fcdbc1031b0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcda3fff700 1 -- 172.21.15.96:0/4226486367 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb0011ce0 con 0x7fcdbc1031b0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc19ff10 con 0x7fcdbc1031b0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcda3fff700 1 -- 172.21.15.96:0/4226486367 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0010d40 con 0x7fcdbc1031b0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.778+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc1a7830 con 0x7fcdbc1031b0 2024-04-25T17:09:26.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.779+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc051190 con 0x7fcdbc1031b0 2024-04-25T17:09:26.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.781+0000 7fcda3fff700 1 -- 172.21.15.96:0/4226486367 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcdb0018020 con 0x7fcdbc1031b0 2024-04-25T17:09:26.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.781+0000 7fcda3fff700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fcda40afc90 0x7fcda40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.781+0000 7fcdbad9d700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fcda40afc90 0x7fcda40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.781+0000 7fcda3fff700 1 -- 172.21.15.96:0/4226486367 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fcdb00de9b0 con 0x7fcdbc1031b0 2024-04-25T17:09:26.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.782+0000 7fcdbad9d700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fcda40afc90 0x7fcda40b21a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcdac003800 tx=0x7fcdac01a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:26.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.785+0000 7fcda3fff700 1 -- 172.21.15.96:0/4226486367 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcdb001f020 con 0x7fcdbc1031b0 2024-04-25T17:09:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 -- 172.21.15.96:0/1733555618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 msgr2=0x7f5d54105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1733555618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d54105740 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009990 tx=0x7f5d48009ca0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 -- 172.21.15.96:0/1733555618 shutdown_connections 2024-04-25T17:09:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1733555618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d5410e7a0 0x7f5d54110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1733555618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d54105740 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1733555618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d54105c80 0x7f5d5410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 -- 172.21.15.96:0/1733555618 >> 172.21.15.96:0/1733555618 conn(0x7f5d540fec90 msgr2=0x7f5d541010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 -- 172.21.15.96:0/1733555618 shutdown_connections 2024-04-25T17:09:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.988+0000 7f5d59abe700 1 -- 172.21.15.96:0/1733555618 wait complete. 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.989+0000 7f5d59abe700 1 Processor -- start 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.989+0000 7f5d59abe700 1 -- start start 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d59abe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d541a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d59abe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d54105c80 0x7f5d541a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d59abe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d5410e7a0 0x7f5d5419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d59abe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541142f0 con 0x7f5d54105330 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d59abe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d54114170 con 0x7f5d5410e7a0 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d59abe700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5d54114470 con 0x7f5d54105c80 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d541a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d541a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59692/0 (socket says 172.21.15.96:59692) 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 -- 172.21.15.96:0/1606893315 learned_addr learned my addr 172.21.15.96:0/1606893315 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d53fff700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d5410e7a0 0x7f5d5419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d52ffd700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d54105c80 0x7f5d541a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 -- 172.21.15.96:0/1606893315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d54105c80 msgr2=0x7f5d541a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d54105c80 0x7f5d541a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 -- 172.21.15.96:0/1606893315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d5410e7a0 msgr2=0x7f5d5419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d5410e7a0 0x7f5d5419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d537fe700 1 -- 172.21.15.96:0/1606893315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5419b230 con 0x7f5d54105330 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.990+0000 7f5d53fff700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d5410e7a0 0x7f5d5419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.991+0000 7f5d537fe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d541a08c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5d480057c0 tx=0x7f5d48009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.991+0000 7f5d50ff9700 1 -- 172.21.15.96:0/1606893315 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48026070 con 0x7f5d54105330 2024-04-25T17:09:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.991+0000 7f5d50ff9700 1 -- 172.21.15.96:0/1606893315 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d48021410 con 0x7f5d54105330 2024-04-25T17:09:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.991+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5419b4c0 con 0x7f5d54105330 2024-04-25T17:09:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.991+0000 7f5d50ff9700 1 -- 172.21.15.96:0/1606893315 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48017650 con 0x7f5d54105330 2024-04-25T17:09:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.991+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d541a7770 con 0x7f5d54105330 2024-04-25T17:09:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.992+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5406bdc0 con 0x7f5d54105330 2024-04-25T17:09:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.994+0000 7f5d50ff9700 1 -- 172.21.15.96:0/1606893315 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5d48029070 con 0x7f5d54105330 2024-04-25T17:09:26.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.994+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcdbc19c0b0 con 0x7fcdbc1031b0 2024-04-25T17:09:26.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.994+0000 7f5d50ff9700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f5d3c0afc10 0x7f5d3c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:26.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.994+0000 7f5d52ffd700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f5d3c0afc10 0x7f5d3c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.994+0000 7f5d50ff9700 1 -- 172.21.15.96:0/1606893315 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f5d480dfdf0 con 0x7f5d54105330 2024-04-25T17:09:26.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.995+0000 7f5d52ffd700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f5d3c0afc10 0x7f5d3c0b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d54102330 tx=0x7f5d44009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:26.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.998+0000 7fcda3fff700 1 -- 172.21.15.96:0/4226486367 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcdb00a7540 con 0x7fcdbc1031b0 2024-04-25T17:09:26.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:26.998+0000 7f5d50ff9700 1 -- 172.21.15.96:0/1606893315 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d48013070 con 0x7f5d54105330 2024-04-25T17:09:26.997 INFO:teuthology.orchestra.run.smithi096.stdout:304942678043 2024-04-25T17:09:26.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.000+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fcda40afc90 msgr2=0x7fcda40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.000+0000 7fcdc108c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fcda40afc90 0x7fcda40b21a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcdac003800 tx=0x7fcdac01a040 comp rx=0 tx=0).stop 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.000+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc1031b0 msgr2=0x7fcdbc19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.000+0000 7fcdc108c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc1031b0 0x7fcdbc19b2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b500 tx=0x7fcdb000b8c0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 shutdown_connections 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc108890 0x7fcdbc19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdbc102860 0x7fcdbc19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fcda40afc90 0x7fcda40b21a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 --2- 172.21.15.96:0/4226486367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdbc1031b0 0x7fcdbc19b2f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 >> 172.21.15.96:0/4226486367 conn(0x7fcdbc0fe3f0 msgr2=0x7fcdbc106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 shutdown_connections 2024-04-25T17:09:27.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.001+0000 7fcdc108c700 1 -- 172.21.15.96:0/4226486367 wait complete. 2024-04-25T17:09:27.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.203+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5d5419bff0 con 0x7f5d54105330 2024-04-25T17:09:27.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.204+0000 7f5d50ff9700 1 -- 172.21.15.96:0/1606893315 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d480a8990 con 0x7f5d54105330 2024-04-25T17:09:27.203 INFO:teuthology.orchestra.run.smithi096.stdout:154618822716 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f5d3c0afc10 msgr2=0x7f5d3c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f5d3c0afc10 0x7f5d3c0b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d54102330 tx=0x7f5d44009260 comp rx=0 tx=0).stop 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 msgr2=0x7f5d541a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d541a08c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5d480057c0 tx=0x7f5d48009580 comp rx=0 tx=0).stop 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 shutdown_connections 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d5410e7a0 0x7f5d5419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f5d3c0afc10 0x7f5d3c0b2120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54105330 0x7f5d541a08c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 --2- 172.21.15.96:0/1606893315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d54105c80 0x7f5d541a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 >> 172.21.15.96:0/1606893315 conn(0x7f5d540fec90 msgr2=0x7f5d54100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 shutdown_connections 2024-04-25T17:09:27.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:27.206+0000 7f5d59abe700 1 -- 172.21.15.96:0/1606893315 wait complete. 2024-04-25T17:09:27.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:27 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/4226486367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T17:09:27.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:27 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/4226486367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T17:09:27.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:27 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/4226486367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T17:09:27.902 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678039 got 304942678043 for osd.11 2024-04-25T17:09:27.903 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:28.349 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037791 got 257698037798 for osd.9 2024-04-25T17:09:28.349 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:28.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:28 smithi096 ceph-mon[29131]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:28.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:28 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1606893315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T17:09:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:28 smithi097 ceph-mon[32602]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:28.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:28 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1606893315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T17:09:28.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:28 smithi171 ceph-mon[33189]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:28.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:28 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1606893315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T17:09:28.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430251 got 206158430257 for osd.7 2024-04-25T17:09:28.965 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:29.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:29 smithi096 ceph-mon[29131]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:29.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:29 smithi097 ceph-mon[32602]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:29.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:29 smithi171 ceph-mon[33189]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:29.868 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822711 got 154618822716 for osd.5 2024-04-25T17:09:29.868 DEBUG:teuthology.parallel:result is None 2024-04-25T17:09:29.868 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T17:09:29.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T17:09:30.142 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.830+0000 7f4410ca5700 1 -- 172.21.15.96:0/963412663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 msgr2=0x7f440c110cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.830+0000 7f4410ca5700 1 --2- 172.21.15.96:0/963412663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c110cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f440400b0a0 tx=0x7f440400b3b0 comp rx=0 tx=0).stop 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.831+0000 7f4410ca5700 1 -- 172.21.15.96:0/963412663 shutdown_connections 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.831+0000 7f4410ca5700 1 --2- 172.21.15.96:0/963412663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c110cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.831+0000 7f4410ca5700 1 --2- 172.21.15.96:0/963412663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f440c105360 0x7f440c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.831+0000 7f4410ca5700 1 --2- 172.21.15.96:0/963412663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440c105cb0 0x7f440c10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.831+0000 7f4410ca5700 1 -- 172.21.15.96:0/963412663 >> 172.21.15.96:0/963412663 conn(0x7f440c0fedb0 msgr2=0x7f440c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.831+0000 7f4410ca5700 1 -- 172.21.15.96:0/963412663 shutdown_connections 2024-04-25T17:09:31.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.831+0000 7f4410ca5700 1 -- 172.21.15.96:0/963412663 wait complete. 2024-04-25T17:09:31.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 Processor -- start 2024-04-25T17:09:31.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 -- start start 2024-04-25T17:09:31.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440c105360 0x7f440c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:31.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f440c105cb0 0x7f440c19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:31.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:31.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f440c114330 con 0x7f440c10e7d0 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1141b0 con 0x7f440c105360 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4410ca5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f440c1144b0 con 0x7f440c105cb0 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f440ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f440ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c1a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58906/0 (socket says 172.21.15.96:58906) 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f440ad9d700 1 -- 172.21.15.96:0/1835805086 learned_addr learned my addr 172.21.15.96:0/1835805086 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f440a59c700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440c105360 0x7f440c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.832+0000 7f4409d9b700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f440c105cb0 0x7f440c19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f440ad9d700 1 -- 172.21.15.96:0/1835805086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f440c105cb0 msgr2=0x7f440c19d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f440ad9d700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f440c105cb0 0x7f440c19d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f440ad9d700 1 -- 172.21.15.96:0/1835805086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440c105360 msgr2=0x7f440c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f440ad9d700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440c105360 0x7f440c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f440ad9d700 1 -- 172.21.15.96:0/1835805086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c1a1cd0 con 0x7f440c10e7d0 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f4409d9b700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f440c105cb0 0x7f440c19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:31.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f440ad9d700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c1a1630 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f440400b830 tx=0x7f4404004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:31.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f43fb7fe700 1 -- 172.21.15.96:0/1835805086 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404021070 con 0x7f440c10e7d0 2024-04-25T17:09:31.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c1a1f60 con 0x7f440c10e7d0 2024-04-25T17:09:31.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f43fb7fe700 1 -- 172.21.15.96:0/1835805086 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404007e30 con 0x7f440c10e7d0 2024-04-25T17:09:31.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f43fb7fe700 1 -- 172.21.15.96:0/1835805086 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040047d0 con 0x7f440c10e7d0 2024-04-25T17:09:31.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.833+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c1a98e0 con 0x7f440c10e7d0 2024-04-25T17:09:31.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.834+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440c0761a0 con 0x7f440c10e7d0 2024-04-25T17:09:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.836+0000 7f43fb7fe700 1 -- 172.21.15.96:0/1835805086 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4404007980 con 0x7f440c10e7d0 2024-04-25T17:09:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.837+0000 7f43fb7fe700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43f40afc10 0x7f43f40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.837+0000 7f440a59c700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43f40afc10 0x7f43f40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.837+0000 7f43fb7fe700 1 -- 172.21.15.96:0/1835805086 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f440400f020 con 0x7f440c10e7d0 2024-04-25T17:09:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.838+0000 7f440a59c700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43f40afc10 0x7f43f40b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4400007be0 tx=0x7f44000073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:31.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:31.842+0000 7f43fb7fe700 1 -- 172.21.15.96:0/1835805086 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f440402a020 con 0x7f440c10e7d0 2024-04-25T17:09:32.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.077+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f440c1036d0 con 0x7f43f40afc10 2024-04-25T17:09:32.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.084+0000 7f43fb7fe700 1 -- 172.21.15.96:0/1835805086 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258922 (secure 0 0 0) 0x7f440c1036d0 con 0x7f43f40afc10 2024-04-25T17:09:32.088 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:32.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43f40afc10 msgr2=0x7f43f40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:32.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43f40afc10 0x7f43f40b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4400007be0 tx=0x7f44000073d0 comp rx=0 tx=0).stop 2024-04-25T17:09:32.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 msgr2=0x7f440c1a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c1a1630 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f440400b830 tx=0x7f4404004ce0 comp rx=0 tx=0).stop 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 shutdown_connections 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f43f40afc10 0x7f43f40b2120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f440c10e7d0 0x7f440c1a1630 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f440c105360 0x7f440c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 --2- 172.21.15.96:0/1835805086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f440c105cb0 0x7f440c19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.088+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 >> 172.21.15.96:0/1835805086 conn(0x7f440c0fedb0 msgr2=0x7f440c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.089+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 shutdown_connections 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:32.089+0000 7f4410ca5700 1 -- 172.21.15.96:0/1835805086 wait complete. 2024-04-25T17:09:32.091 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T17:09:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:32 smithi096 ceph-mon[29131]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:09:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:09:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:32 smithi096 ceph-mon[29131]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.452 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:32 smithi097 ceph-mon[32602]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:32.452 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.452 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.452 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:09:32.452 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:09:32.452 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:32 smithi097 ceph-mon[32602]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.501 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:32 smithi171 ceph-mon[33189]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:09:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:09:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:32 smithi171 ceph-mon[33189]: from='mgr.14148 172.21.15.96:0/1631135124' entity='mgr.x' 2024-04-25T17:09:32.945 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-04-25T17:09:31.081967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652446,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":242,"num_read_kb":211,"num_write":200,"num_write_kb":2235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2952422,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359444,"kb_used_data":10212,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582380,"statfs":{"total":1151940427776,"available":1151572357120,"internally_reserved":0,"allocated":10457088,"data_stored":6832950,"data_compressed":182331,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003122"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.148182+0000","last_change":"2024-04-25T17:08:12.162430+0000","last_active":"2024-04-25T17:09:04.148182+0000","last_peered":"2024-04-25T17:09:04.148182+0000","last_clean":"2024-04-25T17:09:04.148182+0000","last_became_active":"2024-04-25T17:08:12.162052+0000","last_became_peered":"2024-04-25T17:08:12.162052+0000","last_unstale":"2024-04-25T17:09:04.148182+0000","last_undegraded":"2024-04-25T17:09:04.148182+0000","last_fullsized":"2024-04-25T17:09:04.148182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:09.637937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236572+0000","last_change":"2024-04-25T17:08:12.169139+0000","last_active":"2024-04-25T17:09:04.236572+0000","last_peered":"2024-04-25T17:09:04.236572+0000","last_clean":"2024-04-25T17:09:04.236572+0000","last_became_active":"2024-04-25T17:08:12.168715+0000","last_became_peered":"2024-04-25T17:08:12.168715+0000","last_unstale":"2024-04-25T17:09:04.236572+0000","last_undegraded":"2024-04-25T17:09:04.236572+0000","last_fullsized":"2024-04-25T17:09:04.236572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:57.357750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236493+0000","last_change":"2024-04-25T17:08:12.159141+0000","last_active":"2024-04-25T17:09:04.236493+0000","last_peered":"2024-04-25T17:09:04.236493+0000","last_clean":"2024-04-25T17:09:04.236493+0000","last_became_active":"2024-04-25T17:08:12.158796+0000","last_became_peered":"2024-04-25T17:08:12.158796+0000","last_unstale":"2024-04-25T17:09:04.236493+0000","last_undegraded":"2024-04-25T17:09:04.236493+0000","last_fullsized":"2024-04-25T17:09:04.236493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:55.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145879+0000","last_change":"2024-04-25T17:08:12.160464+0000","last_active":"2024-04-25T17:09:04.145879+0000","last_peered":"2024-04-25T17:09:04.145879+0000","last_clean":"2024-04-25T17:09:04.145879+0000","last_became_active":"2024-04-25T17:08:12.160118+0000","last_became_peered":"2024-04-25T17:08:12.160118+0000","last_unstale":"2024-04-25T17:09:04.145879+0000","last_undegraded":"2024-04-25T17:09:04.145879+0000","last_fullsized":"2024-04-25T17:09:04.145879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:59:06.465964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149631+0000","last_change":"2024-04-25T17:08:12.174591+0000","last_active":"2024-04-25T17:09:04.149631+0000","last_peered":"2024-04-25T17:09:04.149631+0000","last_clean":"2024-04-25T17:09:04.149631+0000","last_became_active":"2024-04-25T17:08:12.174173+0000","last_became_peered":"2024-04-25T17:08:12.174173+0000","last_unstale":"2024-04-25T17:09:04.149631+0000","last_undegraded":"2024-04-25T17:09:04.149631+0000","last_fullsized":"2024-04-25T17:09:04.149631+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:12.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147188+0000","last_change":"2024-04-25T17:08:12.164230+0000","last_active":"2024-04-25T17:09:04.147188+0000","last_peered":"2024-04-25T17:09:04.147188+0000","last_clean":"2024-04-25T17:09:04.147188+0000","last_became_active":"2024-04-25T17:08:12.163868+0000","last_became_peered":"2024-04-25T17:08:12.163868+0000","last_unstale":"2024-04-25T17:09:04.147188+0000","last_undegraded":"2024-04-25T17:09:04.147188+0000","last_fullsized":"2024-04-25T17:09:04.147188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:12.604164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149993+0000","last_change":"2024-04-25T17:08:12.161627+0000","last_active":"2024-04-25T17:09:04.149993+0000","last_peered":"2024-04-25T17:09:04.149993+0000","last_clean":"2024-04-25T17:09:04.149993+0000","last_became_active":"2024-04-25T17:08:12.161146+0000","last_became_peered":"2024-04-25T17:08:12.161146+0000","last_unstale":"2024-04-25T17:09:04.149993+0000","last_undegraded":"2024-04-25T17:09:04.149993+0000","last_fullsized":"2024-04-25T17:09:04.149993+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.972053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236010+0000","last_change":"2024-04-25T17:08:12.165512+0000","last_active":"2024-04-25T17:09:04.236010+0000","last_peered":"2024-04-25T17:09:04.236010+0000","last_clean":"2024-04-25T17:09:04.236010+0000","last_became_active":"2024-04-25T17:08:12.164018+0000","last_became_peered":"2024-04-25T17:08:12.164018+0000","last_unstale":"2024-04-25T17:09:04.236010+0000","last_undegraded":"2024-04-25T17:09:04.236010+0000","last_fullsized":"2024-04-25T17:09:04.236010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:28.075786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.237015+0000","last_change":"2024-04-25T17:08:12.173313+0000","last_active":"2024-04-25T17:09:04.237015+0000","last_peered":"2024-04-25T17:09:04.237015+0000","last_clean":"2024-04-25T17:09:04.237015+0000","last_became_active":"2024-04-25T17:08:12.171269+0000","last_became_peered":"2024-04-25T17:08:12.171269+0000","last_unstale":"2024-04-25T17:09:04.237015+0000","last_undegraded":"2024-04-25T17:09:04.237015+0000","last_fullsized":"2024-04-25T17:09:04.237015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:39.565973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.156007+0000","last_change":"2024-04-25T17:09:04.156007+0000","last_active":"2024-04-25T17:09:04.156007+0000","last_peered":"2024-04-25T17:09:04.156007+0000","last_clean":"2024-04-25T17:09:04.156007+0000","last_became_active":"2024-04-25T17:09:04.155636+0000","last_became_peered":"2024-04-25T17:09:04.155636+0000","last_unstale":"2024-04-25T17:09:04.156007+0000","last_undegraded":"2024-04-25T17:09:04.156007+0000","last_fullsized":"2024-04-25T17:09:04.156007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:30.993817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150301+0000","last_change":"2024-04-25T17:08:12.160435+0000","last_active":"2024-04-25T17:09:04.150301+0000","last_peered":"2024-04-25T17:09:04.150301+0000","last_clean":"2024-04-25T17:09:04.150301+0000","last_became_active":"2024-04-25T17:08:12.160085+0000","last_became_peered":"2024-04-25T17:08:12.160085+0000","last_unstale":"2024-04-25T17:09:04.150301+0000","last_undegraded":"2024-04-25T17:09:04.150301+0000","last_fullsized":"2024-04-25T17:09:04.150301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.709567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145796+0000","last_change":"2024-04-25T17:08:12.173990+0000","last_active":"2024-04-25T17:09:04.145796+0000","last_peered":"2024-04-25T17:09:04.145796+0000","last_clean":"2024-04-25T17:09:04.145796+0000","last_became_active":"2024-04-25T17:08:12.172838+0000","last_became_peered":"2024-04-25T17:08:12.172838+0000","last_unstale":"2024-04-25T17:09:04.145796+0000","last_undegraded":"2024-04-25T17:09:04.145796+0000","last_fullsized":"2024-04-25T17:09:04.145796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:00:56.720758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145381+0000","last_change":"2024-04-25T17:08:12.167579+0000","last_active":"2024-04-25T17:09:04.145381+0000","last_peered":"2024-04-25T17:09:04.145381+0000","last_clean":"2024-04-25T17:09:04.145381+0000","last_became_active":"2024-04-25T17:08:12.166583+0000","last_became_peered":"2024-04-25T17:08:12.166583+0000","last_unstale":"2024-04-25T17:09:04.145381+0000","last_undegraded":"2024-04-25T17:09:04.145381+0000","last_fullsized":"2024-04-25T17:09:04.145381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:44.081814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236175+0000","last_change":"2024-04-25T17:08:12.161263+0000","last_active":"2024-04-25T17:09:04.236175+0000","last_peered":"2024-04-25T17:09:04.236175+0000","last_clean":"2024-04-25T17:09:04.236175+0000","last_became_active":"2024-04-25T17:08:12.160078+0000","last_became_peered":"2024-04-25T17:08:12.160078+0000","last_unstale":"2024-04-25T17:09:04.236175+0000","last_undegraded":"2024-04-25T17:09:04.236175+0000","last_fullsized":"2024-04-25T17:09:04.236175+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:56.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236170+0000","last_change":"2024-04-25T17:08:12.167318+0000","last_active":"2024-04-25T17:09:04.236170+0000","last_peered":"2024-04-25T17:09:04.236170+0000","last_clean":"2024-04-25T17:09:04.236170+0000","last_became_active":"2024-04-25T17:08:12.166964+0000","last_became_peered":"2024-04-25T17:08:12.166964+0000","last_unstale":"2024-04-25T17:09:04.236170+0000","last_undegraded":"2024-04-25T17:09:04.236170+0000","last_fullsized":"2024-04-25T17:09:04.236170+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:25.967818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236449+0000","last_change":"2024-04-25T17:08:12.166417+0000","last_active":"2024-04-25T17:09:04.236449+0000","last_peered":"2024-04-25T17:09:04.236449+0000","last_clean":"2024-04-25T17:09:04.236449+0000","last_became_active":"2024-04-25T17:08:12.165928+0000","last_became_peered":"2024-04-25T17:08:12.165928+0000","last_unstale":"2024-04-25T17:09:04.236449+0000","last_undegraded":"2024-04-25T17:09:04.236449+0000","last_fullsized":"2024-04-25T17:09:04.236449+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:24:22.359814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.148262+0000","last_change":"2024-04-25T17:08:12.164952+0000","last_active":"2024-04-25T17:09:04.148262+0000","last_peered":"2024-04-25T17:09:04.148262+0000","last_clean":"2024-04-25T17:09:04.148262+0000","last_became_active":"2024-04-25T17:08:12.164595+0000","last_became_peered":"2024-04-25T17:08:12.164595+0000","last_unstale":"2024-04-25T17:09:04.148262+0000","last_undegraded":"2024-04-25T17:09:04.148262+0000","last_fullsized":"2024-04-25T17:09:04.148262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:45.020512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149682+0000","last_change":"2024-04-25T17:08:12.153583+0000","last_active":"2024-04-25T17:09:04.149682+0000","last_peered":"2024-04-25T17:09:04.149682+0000","last_clean":"2024-04-25T17:09:04.149682+0000","last_became_active":"2024-04-25T17:08:12.153070+0000","last_became_peered":"2024-04-25T17:08:12.153070+0000","last_unstale":"2024-04-25T17:09:04.149682+0000","last_undegraded":"2024-04-25T17:09:04.149682+0000","last_fullsized":"2024-04-25T17:09:04.149682+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:34.151002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151671+0000","last_change":"2024-04-25T17:08:12.166837+0000","last_active":"2024-04-25T17:09:04.151671+0000","last_peered":"2024-04-25T17:09:04.151671+0000","last_clean":"2024-04-25T17:09:04.151671+0000","last_became_active":"2024-04-25T17:08:12.166477+0000","last_became_peered":"2024-04-25T17:08:12.166477+0000","last_unstale":"2024-04-25T17:09:04.151671+0000","last_undegraded":"2024-04-25T17:09:04.151671+0000","last_fullsized":"2024-04-25T17:09:04.151671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:50.492333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.241787+0000","last_change":"2024-04-25T17:09:04.241787+0000","last_active":"2024-04-25T17:09:04.241787+0000","last_peered":"2024-04-25T17:09:04.241787+0000","last_clean":"2024-04-25T17:09:04.241787+0000","last_became_active":"2024-04-25T17:09:04.237983+0000","last_became_peered":"2024-04-25T17:09:04.237983+0000","last_unstale":"2024-04-25T17:09:04.241787+0000","last_undegraded":"2024-04-25T17:09:04.241787+0000","last_fullsized":"2024-04-25T17:09:04.241787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:02.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236906+0000","last_change":"2024-04-25T17:08:12.178117+0000","last_active":"2024-04-25T17:09:04.236906+0000","last_peered":"2024-04-25T17:09:04.236906+0000","last_clean":"2024-04-25T17:09:04.236906+0000","last_became_active":"2024-04-25T17:08:12.177767+0000","last_became_peered":"2024-04-25T17:08:12.177767+0000","last_unstale":"2024-04-25T17:09:04.236906+0000","last_undegraded":"2024-04-25T17:09:04.236906+0000","last_fullsized":"2024-04-25T17:09:04.236906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:34.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.237070+0000","last_change":"2024-04-25T17:08:11.153996+0000","last_active":"2024-04-25T17:09:04.237070+0000","last_peered":"2024-04-25T17:09:04.237070+0000","last_clean":"2024-04-25T17:09:04.237070+0000","last_became_active":"2024-04-25T17:08:11.152993+0000","last_became_peered":"2024-04-25T17:08:11.152993+0000","last_unstale":"2024-04-25T17:09:04.237070+0000","last_undegraded":"2024-04-25T17:09:04.237070+0000","last_fullsized":"2024-04-25T17:09:04.237070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:33.116616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236667+0000","last_change":"2024-04-25T17:08:12.163707+0000","last_active":"2024-04-25T17:09:04.236667+0000","last_peered":"2024-04-25T17:09:04.236667+0000","last_clean":"2024-04-25T17:09:04.236667+0000","last_became_active":"2024-04-25T17:08:12.161868+0000","last_became_peered":"2024-04-25T17:08:12.161868+0000","last_unstale":"2024-04-25T17:09:04.236667+0000","last_undegraded":"2024-04-25T17:09:04.236667+0000","last_fullsized":"2024-04-25T17:09:04.236667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.333036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.160023+0000","last_change":"2024-04-25T17:09:04.160023+0000","last_active":"2024-04-25T17:09:04.160023+0000","last_peered":"2024-04-25T17:09:04.160023+0000","last_clean":"2024-04-25T17:09:04.160023+0000","last_became_active":"2024-04-25T17:09:04.154073+0000","last_became_peered":"2024-04-25T17:09:04.154073+0000","last_unstale":"2024-04-25T17:09:04.160023+0000","last_undegraded":"2024-04-25T17:09:04.160023+0000","last_fullsized":"2024-04-25T17:09:04.160023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:59.181852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236619+0000","last_change":"2024-04-25T17:08:12.170278+0000","last_active":"2024-04-25T17:09:04.236619+0000","last_peered":"2024-04-25T17:09:04.236619+0000","last_clean":"2024-04-25T17:09:04.236619+0000","last_became_active":"2024-04-25T17:08:12.169219+0000","last_became_peered":"2024-04-25T17:08:12.169219+0000","last_unstale":"2024-04-25T17:09:04.236619+0000","last_undegraded":"2024-04-25T17:09:04.236619+0000","last_fullsized":"2024-04-25T17:09:04.236619+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:35.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149829+0000","last_change":"2024-04-25T17:08:11.153167+0000","last_active":"2024-04-25T17:09:04.149829+0000","last_peered":"2024-04-25T17:09:04.149829+0000","last_clean":"2024-04-25T17:09:04.149829+0000","last_became_active":"2024-04-25T17:08:11.152772+0000","last_became_peered":"2024-04-25T17:08:11.152772+0000","last_unstale":"2024-04-25T17:09:04.149829+0000","last_undegraded":"2024-04-25T17:09:04.149829+0000","last_fullsized":"2024-04-25T17:09:04.149829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:57.198963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.146054+0000","last_change":"2024-04-25T17:08:12.163530+0000","last_active":"2024-04-25T17:09:04.146054+0000","last_peered":"2024-04-25T17:09:04.146054+0000","last_clean":"2024-04-25T17:09:04.146054+0000","last_became_active":"2024-04-25T17:08:12.163179+0000","last_became_peered":"2024-04-25T17:08:12.163179+0000","last_unstale":"2024-04-25T17:09:04.146054+0000","last_undegraded":"2024-04-25T17:09:04.146054+0000","last_fullsized":"2024-04-25T17:09:04.146054+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:59.536274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.155403+0000","last_change":"2024-04-25T17:09:04.154296+0000","last_active":"2024-04-25T17:09:04.155403+0000","last_peered":"2024-04-25T17:09:04.155403+0000","last_clean":"2024-04-25T17:09:04.155403+0000","last_became_active":"2024-04-25T17:09:04.153661+0000","last_became_peered":"2024-04-25T17:09:04.153661+0000","last_unstale":"2024-04-25T17:09:04.155403+0000","last_undegraded":"2024-04-25T17:09:04.155403+0000","last_fullsized":"2024-04-25T17:09:04.155403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.170563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149797+0000","last_change":"2024-04-25T17:08:12.167584+0000","last_active":"2024-04-25T17:09:04.149797+0000","last_peered":"2024-04-25T17:09:04.149797+0000","last_clean":"2024-04-25T17:09:04.149797+0000","last_became_active":"2024-04-25T17:08:12.166717+0000","last_became_peered":"2024-04-25T17:08:12.166717+0000","last_unstale":"2024-04-25T17:09:04.149797+0000","last_undegraded":"2024-04-25T17:09:04.149797+0000","last_fullsized":"2024-04-25T17:09:04.149797+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:38.047671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149330+0000","last_change":"2024-04-25T17:09:04.149330+0000","last_active":"2024-04-25T17:09:04.149330+0000","last_peered":"2024-04-25T17:09:04.149330+0000","last_clean":"2024-04-25T17:09:04.149330+0000","last_became_active":"2024-04-25T17:09:04.148884+0000","last_became_peered":"2024-04-25T17:09:04.148884+0000","last_unstale":"2024-04-25T17:09:04.149330+0000","last_undegraded":"2024-04-25T17:09:04.149330+0000","last_fullsized":"2024-04-25T17:09:04.149330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:25.787948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145804+0000","last_change":"2024-04-25T17:08:12.170409+0000","last_active":"2024-04-25T17:09:04.145804+0000","last_peered":"2024-04-25T17:09:04.145804+0000","last_clean":"2024-04-25T17:09:04.145804+0000","last_became_active":"2024-04-25T17:08:12.169510+0000","last_became_peered":"2024-04-25T17:08:12.169510+0000","last_unstale":"2024-04-25T17:09:04.145804+0000","last_undegraded":"2024-04-25T17:09:04.145804+0000","last_fullsized":"2024-04-25T17:09:04.145804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.310486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.144829+0000","last_change":"2024-04-25T17:08:11.150980+0000","last_active":"2024-04-25T17:09:04.144829+0000","last_peered":"2024-04-25T17:09:04.144829+0000","last_clean":"2024-04-25T17:09:04.144829+0000","last_became_active":"2024-04-25T17:08:11.150580+0000","last_became_peered":"2024-04-25T17:08:11.150580+0000","last_unstale":"2024-04-25T17:09:04.144829+0000","last_undegraded":"2024-04-25T17:09:04.144829+0000","last_fullsized":"2024-04-25T17:09:04.144829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:59.539673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236048+0000","last_change":"2024-04-25T17:08:12.171013+0000","last_active":"2024-04-25T17:09:04.236048+0000","last_peered":"2024-04-25T17:09:04.236048+0000","last_clean":"2024-04-25T17:09:04.236048+0000","last_became_active":"2024-04-25T17:08:12.169868+0000","last_became_peered":"2024-04-25T17:08:12.169868+0000","last_unstale":"2024-04-25T17:09:04.236048+0000","last_undegraded":"2024-04-25T17:09:04.236048+0000","last_fullsized":"2024-04-25T17:09:04.236048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:22.418990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145580+0000","last_change":"2024-04-25T17:08:11.157618+0000","last_active":"2024-04-25T17:09:04.145580+0000","last_peered":"2024-04-25T17:09:04.145580+0000","last_clean":"2024-04-25T17:09:04.145580+0000","last_became_active":"2024-04-25T17:08:11.157052+0000","last_became_peered":"2024-04-25T17:08:11.157052+0000","last_unstale":"2024-04-25T17:09:04.145580+0000","last_undegraded":"2024-04-25T17:09:04.145580+0000","last_fullsized":"2024-04-25T17:09:04.145580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:06.032163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149023+0000","last_change":"2024-04-25T17:08:12.151093+0000","last_active":"2024-04-25T17:09:04.149023+0000","last_peered":"2024-04-25T17:09:04.149023+0000","last_clean":"2024-04-25T17:09:04.149023+0000","last_became_active":"2024-04-25T17:08:12.150699+0000","last_became_peered":"2024-04-25T17:08:12.150699+0000","last_unstale":"2024-04-25T17:09:04.149023+0000","last_undegraded":"2024-04-25T17:09:04.149023+0000","last_fullsized":"2024-04-25T17:09:04.149023+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:27.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.148011+0000","last_change":"2024-04-25T17:08:11.157673+0000","last_active":"2024-04-25T17:09:04.148011+0000","last_peered":"2024-04-25T17:09:04.148011+0000","last_clean":"2024-04-25T17:09:04.148011+0000","last_became_active":"2024-04-25T17:08:11.157089+0000","last_became_peered":"2024-04-25T17:08:11.157089+0000","last_unstale":"2024-04-25T17:09:04.148011+0000","last_undegraded":"2024-04-25T17:09:04.148011+0000","last_fullsized":"2024-04-25T17:09:04.148011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:47.586003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236887+0000","last_change":"2024-04-25T17:08:12.170666+0000","last_active":"2024-04-25T17:09:04.236887+0000","last_peered":"2024-04-25T17:09:04.236887+0000","last_clean":"2024-04-25T17:09:04.236887+0000","last_became_active":"2024-04-25T17:08:12.169607+0000","last_became_peered":"2024-04-25T17:08:12.169607+0000","last_unstale":"2024-04-25T17:09:04.236887+0000","last_undegraded":"2024-04-25T17:09:04.236887+0000","last_fullsized":"2024-04-25T17:09:04.236887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149916+0000","last_change":"2024-04-25T17:08:11.153407+0000","last_active":"2024-04-25T17:09:04.149916+0000","last_peered":"2024-04-25T17:09:04.149916+0000","last_clean":"2024-04-25T17:09:04.149916+0000","last_became_active":"2024-04-25T17:08:11.152932+0000","last_became_peered":"2024-04-25T17:08:11.152932+0000","last_unstale":"2024-04-25T17:09:04.149916+0000","last_undegraded":"2024-04-25T17:09:04.149916+0000","last_fullsized":"2024-04-25T17:09:04.149916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:06.712620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236617+0000","last_change":"2024-04-25T17:08:12.173788+0000","last_active":"2024-04-25T17:09:04.236617+0000","last_peered":"2024-04-25T17:09:04.236617+0000","last_clean":"2024-04-25T17:09:04.236617+0000","last_became_active":"2024-04-25T17:08:12.172714+0000","last_became_peered":"2024-04-25T17:08:12.172714+0000","last_unstale":"2024-04-25T17:09:04.236617+0000","last_undegraded":"2024-04-25T17:09:04.236617+0000","last_fullsized":"2024-04-25T17:09:04.236617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:18.892187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.146168+0000","last_change":"2024-04-25T17:08:11.159802+0000","last_active":"2024-04-25T17:09:04.146168+0000","last_peered":"2024-04-25T17:09:04.146168+0000","last_clean":"2024-04-25T17:09:04.146168+0000","last_became_active":"2024-04-25T17:08:11.159431+0000","last_became_peered":"2024-04-25T17:08:11.159431+0000","last_unstale":"2024-04-25T17:09:04.146168+0000","last_undegraded":"2024-04-25T17:09:04.146168+0000","last_fullsized":"2024-04-25T17:09:04.146168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236738+0000","last_change":"2024-04-25T17:08:12.162703+0000","last_active":"2024-04-25T17:09:04.236738+0000","last_peered":"2024-04-25T17:09:04.236738+0000","last_clean":"2024-04-25T17:09:04.236738+0000","last_became_active":"2024-04-25T17:08:12.162361+0000","last_became_peered":"2024-04-25T17:08:12.162361+0000","last_unstale":"2024-04-25T17:09:04.236738+0000","last_undegraded":"2024-04-25T17:09:04.236738+0000","last_fullsized":"2024-04-25T17:09:04.236738+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:51.262319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'26","reported_seq":200,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:28.189322+0000","last_change":"2024-04-25T17:08:11.153938+0000","last_active":"2024-04-25T17:09:28.189322+0000","last_peered":"2024-04-25T17:09:28.189322+0000","last_clean":"2024-04-25T17:09:28.189322+0000","last_became_active":"2024-04-25T17:08:11.153591+0000","last_became_peered":"2024-04-25T17:08:11.153591+0000","last_unstale":"2024-04-25T17:09:28.189322+0000","last_undegraded":"2024-04-25T17:09:28.189322+0000","last_fullsized":"2024-04-25T17:09:28.189322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":155,"num_read_kb":136,"num_write":26,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149884+0000","last_change":"2024-04-25T17:08:12.156045+0000","last_active":"2024-04-25T17:09:04.149884+0000","last_peered":"2024-04-25T17:09:04.149884+0000","last_clean":"2024-04-25T17:09:04.149884+0000","last_became_active":"2024-04-25T17:08:12.155657+0000","last_became_peered":"2024-04-25T17:08:12.155657+0000","last_unstale":"2024-04-25T17:09:04.149884+0000","last_undegraded":"2024-04-25T17:09:04.149884+0000","last_fullsized":"2024-04-25T17:09:04.149884+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151133+0000","last_change":"2024-04-25T17:08:11.162009+0000","last_active":"2024-04-25T17:09:04.151133+0000","last_peered":"2024-04-25T17:09:04.151133+0000","last_clean":"2024-04-25T17:09:04.151133+0000","last_became_active":"2024-04-25T17:08:11.161574+0000","last_became_peered":"2024-04-25T17:08:11.161574+0000","last_unstale":"2024-04-25T17:09:04.151133+0000","last_undegraded":"2024-04-25T17:09:04.151133+0000","last_fullsized":"2024-04-25T17:09:04.151133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:16.760355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.144978+0000","last_change":"2024-04-25T17:08:12.159522+0000","last_active":"2024-04-25T17:09:04.144978+0000","last_peered":"2024-04-25T17:09:04.144978+0000","last_clean":"2024-04-25T17:09:04.144978+0000","last_became_active":"2024-04-25T17:08:12.159036+0000","last_became_peered":"2024-04-25T17:08:12.159036+0000","last_unstale":"2024-04-25T17:09:04.144978+0000","last_undegraded":"2024-04-25T17:09:04.144978+0000","last_fullsized":"2024-04-25T17:09:04.144978+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:32.044488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145694+0000","last_change":"2024-04-25T17:08:11.162487+0000","last_active":"2024-04-25T17:09:04.145694+0000","last_peered":"2024-04-25T17:09:04.145694+0000","last_clean":"2024-04-25T17:09:04.145694+0000","last_became_active":"2024-04-25T17:08:11.161961+0000","last_became_peered":"2024-04-25T17:08:11.161961+0000","last_unstale":"2024-04-25T17:09:04.145694+0000","last_undegraded":"2024-04-25T17:09:04.145694+0000","last_fullsized":"2024-04-25T17:09:04.145694+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:02.643683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150472+0000","last_change":"2024-04-25T17:08:12.164107+0000","last_active":"2024-04-25T17:09:04.150472+0000","last_peered":"2024-04-25T17:09:04.150472+0000","last_clean":"2024-04-25T17:09:04.150472+0000","last_became_active":"2024-04-25T17:08:12.163741+0000","last_became_peered":"2024-04-25T17:08:12.163741+0000","last_unstale":"2024-04-25T17:09:04.150472+0000","last_undegraded":"2024-04-25T17:09:04.150472+0000","last_fullsized":"2024-04-25T17:09:04.150472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.552833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145227+0000","last_change":"2024-04-25T17:08:11.153383+0000","last_active":"2024-04-25T17:09:04.145227+0000","last_peered":"2024-04-25T17:09:04.145227+0000","last_clean":"2024-04-25T17:09:04.145227+0000","last_became_active":"2024-04-25T17:08:11.152573+0000","last_became_peered":"2024-04-25T17:08:11.152573+0000","last_unstale":"2024-04-25T17:09:04.145227+0000","last_undegraded":"2024-04-25T17:09:04.145227+0000","last_fullsized":"2024-04-25T17:09:04.145227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:13.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149724+0000","last_change":"2024-04-25T17:08:12.165939+0000","last_active":"2024-04-25T17:09:04.149724+0000","last_peered":"2024-04-25T17:09:04.149724+0000","last_clean":"2024-04-25T17:09:04.149724+0000","last_became_active":"2024-04-25T17:08:12.165569+0000","last_became_peered":"2024-04-25T17:08:12.165569+0000","last_unstale":"2024-04-25T17:09:04.149724+0000","last_undegraded":"2024-04-25T17:09:04.149724+0000","last_fullsized":"2024-04-25T17:09:04.149724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:15:16.230457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149511+0000","last_change":"2024-04-25T17:08:11.158927+0000","last_active":"2024-04-25T17:09:04.149511+0000","last_peered":"2024-04-25T17:09:04.149511+0000","last_clean":"2024-04-25T17:09:04.149511+0000","last_became_active":"2024-04-25T17:08:11.158528+0000","last_became_peered":"2024-04-25T17:08:11.158528+0000","last_unstale":"2024-04-25T17:09:04.149511+0000","last_undegraded":"2024-04-25T17:09:04.149511+0000","last_fullsized":"2024-04-25T17:09:04.149511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.308627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145716+0000","last_change":"2024-04-25T17:08:12.160776+0000","last_active":"2024-04-25T17:09:04.145716+0000","last_peered":"2024-04-25T17:09:04.145716+0000","last_clean":"2024-04-25T17:09:04.145716+0000","last_became_active":"2024-04-25T17:08:12.160409+0000","last_became_peered":"2024-04-25T17:08:12.160409+0000","last_unstale":"2024-04-25T17:09:04.145716+0000","last_undegraded":"2024-04-25T17:09:04.145716+0000","last_fullsized":"2024-04-25T17:09:04.145716+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:51.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150280+0000","last_change":"2024-04-25T17:08:11.153214+0000","last_active":"2024-04-25T17:09:04.150280+0000","last_peered":"2024-04-25T17:09:04.150280+0000","last_clean":"2024-04-25T17:09:04.150280+0000","last_became_active":"2024-04-25T17:08:11.152728+0000","last_became_peered":"2024-04-25T17:08:11.152728+0000","last_unstale":"2024-04-25T17:09:04.150280+0000","last_undegraded":"2024-04-25T17:09:04.150280+0000","last_fullsized":"2024-04-25T17:09:04.150280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.338549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151034+0000","last_change":"2024-04-25T17:09:04.151034+0000","last_active":"2024-04-25T17:09:04.151034+0000","last_peered":"2024-04-25T17:09:04.151034+0000","last_clean":"2024-04-25T17:09:04.151034+0000","last_became_active":"2024-04-25T17:09:04.150605+0000","last_became_peered":"2024-04-25T17:09:04.150605+0000","last_unstale":"2024-04-25T17:09:04.151034+0000","last_undegraded":"2024-04-25T17:09:04.151034+0000","last_fullsized":"2024-04-25T17:09:04.151034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:32.853957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149135+0000","last_change":"2024-04-25T17:08:11.158995+0000","last_active":"2024-04-25T17:09:04.149135+0000","last_peered":"2024-04-25T17:09:04.149135+0000","last_clean":"2024-04-25T17:09:04.149135+0000","last_became_active":"2024-04-25T17:08:11.158644+0000","last_became_peered":"2024-04-25T17:08:11.158644+0000","last_unstale":"2024-04-25T17:09:04.149135+0000","last_undegraded":"2024-04-25T17:09:04.149135+0000","last_fullsized":"2024-04-25T17:09:04.149135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:22.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147959+0000","last_change":"2024-04-25T17:08:12.161635+0000","last_active":"2024-04-25T17:09:04.147959+0000","last_peered":"2024-04-25T17:09:04.147959+0000","last_clean":"2024-04-25T17:09:04.147959+0000","last_became_active":"2024-04-25T17:08:12.153728+0000","last_became_peered":"2024-04-25T17:08:12.153728+0000","last_unstale":"2024-04-25T17:09:04.147959+0000","last_undegraded":"2024-04-25T17:09:04.147959+0000","last_fullsized":"2024-04-25T17:09:04.147959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:02.003550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150239+0000","last_change":"2024-04-25T17:08:11.150490+0000","last_active":"2024-04-25T17:09:04.150239+0000","last_peered":"2024-04-25T17:09:04.150239+0000","last_clean":"2024-04-25T17:09:04.150239+0000","last_became_active":"2024-04-25T17:08:11.148391+0000","last_became_peered":"2024-04-25T17:08:11.148391+0000","last_unstale":"2024-04-25T17:09:04.150239+0000","last_undegraded":"2024-04-25T17:09:04.150239+0000","last_fullsized":"2024-04-25T17:09:04.150239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:40.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.146994+0000","last_change":"2024-04-25T17:08:12.168136+0000","last_active":"2024-04-25T17:09:04.146994+0000","last_peered":"2024-04-25T17:09:04.146994+0000","last_clean":"2024-04-25T17:09:04.146994+0000","last_became_active":"2024-04-25T17:08:12.167786+0000","last_became_peered":"2024-04-25T17:08:12.167786+0000","last_unstale":"2024-04-25T17:09:04.146994+0000","last_undegraded":"2024-04-25T17:09:04.146994+0000","last_fullsized":"2024-04-25T17:09:04.146994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:22.869503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145916+0000","last_change":"2024-04-25T17:08:12.159952+0000","last_active":"2024-04-25T17:09:04.145916+0000","last_peered":"2024-04-25T17:09:04.145916+0000","last_clean":"2024-04-25T17:09:04.145916+0000","last_became_active":"2024-04-25T17:08:12.159429+0000","last_became_peered":"2024-04-25T17:08:12.159429+0000","last_unstale":"2024-04-25T17:09:04.145916+0000","last_undegraded":"2024-04-25T17:09:04.145916+0000","last_fullsized":"2024-04-25T17:09:04.145916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:13:30.521084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.144707+0000","last_change":"2024-04-25T17:08:11.149085+0000","last_active":"2024-04-25T17:09:04.144707+0000","last_peered":"2024-04-25T17:09:04.144707+0000","last_clean":"2024-04-25T17:09:04.144707+0000","last_became_active":"2024-04-25T17:08:11.148712+0000","last_became_peered":"2024-04-25T17:08:11.148712+0000","last_unstale":"2024-04-25T17:09:04.144707+0000","last_undegraded":"2024-04-25T17:09:04.144707+0000","last_fullsized":"2024-04-25T17:09:04.144707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:12.764034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236861+0000","last_change":"2024-04-25T17:08:12.178154+0000","last_active":"2024-04-25T17:09:04.236861+0000","last_peered":"2024-04-25T17:09:04.236861+0000","last_clean":"2024-04-25T17:09:04.236861+0000","last_became_active":"2024-04-25T17:08:12.177773+0000","last_became_peered":"2024-04-25T17:08:12.177773+0000","last_unstale":"2024-04-25T17:09:04.236861+0000","last_undegraded":"2024-04-25T17:09:04.236861+0000","last_fullsized":"2024-04-25T17:09:04.236861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.962410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145510+0000","last_change":"2024-04-25T17:08:11.152932+0000","last_active":"2024-04-25T17:09:04.145510+0000","last_peered":"2024-04-25T17:09:04.145510+0000","last_clean":"2024-04-25T17:09:04.145510+0000","last_became_active":"2024-04-25T17:08:11.152530+0000","last_became_peered":"2024-04-25T17:08:11.152530+0000","last_unstale":"2024-04-25T17:09:04.145510+0000","last_undegraded":"2024-04-25T17:09:04.145510+0000","last_fullsized":"2024-04-25T17:09:04.145510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149811+0000","last_change":"2024-04-25T17:06:55.951326+0000","last_active":"2024-04-25T17:09:04.149811+0000","last_peered":"2024-04-25T17:09:04.149811+0000","last_clean":"2024-04-25T17:09:04.149811+0000","last_became_active":"2024-04-25T17:06:55.946641+0000","last_became_peered":"2024-04-25T17:06:55.946641+0000","last_unstale":"2024-04-25T17:09:04.149811+0000","last_undegraded":"2024-04-25T17:09:04.149811+0000","last_fullsized":"2024-04-25T17:09:04.149811+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_clean_scrub_stamp":"2024-04-25T17:03:27.394276+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:58.308843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.152927+0000","last_change":"2024-04-25T17:09:04.152927+0000","last_active":"2024-04-25T17:09:04.152927+0000","last_peered":"2024-04-25T17:09:04.152927+0000","last_clean":"2024-04-25T17:09:04.152927+0000","last_became_active":"2024-04-25T17:09:04.152541+0000","last_became_peered":"2024-04-25T17:09:04.152541+0000","last_unstale":"2024-04-25T17:09:04.152927+0000","last_undegraded":"2024-04-25T17:09:04.152927+0000","last_fullsized":"2024-04-25T17:09:04.152927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:30.076215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150651+0000","last_change":"2024-04-25T17:08:11.155941+0000","last_active":"2024-04-25T17:09:04.150651+0000","last_peered":"2024-04-25T17:09:04.150651+0000","last_clean":"2024-04-25T17:09:04.150651+0000","last_became_active":"2024-04-25T17:08:11.155381+0000","last_became_peered":"2024-04-25T17:08:11.155381+0000","last_unstale":"2024-04-25T17:09:04.150651+0000","last_undegraded":"2024-04-25T17:09:04.150651+0000","last_fullsized":"2024-04-25T17:09:04.150651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:28.423462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150036+0000","last_change":"2024-04-25T17:08:11.158960+0000","last_active":"2024-04-25T17:09:04.150036+0000","last_peered":"2024-04-25T17:09:04.150036+0000","last_clean":"2024-04-25T17:09:04.150036+0000","last_became_active":"2024-04-25T17:08:11.158609+0000","last_became_peered":"2024-04-25T17:08:11.158609+0000","last_unstale":"2024-04-25T17:09:04.150036+0000","last_undegraded":"2024-04-25T17:09:04.150036+0000","last_fullsized":"2024-04-25T17:09:04.150036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:19.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145612+0000","last_change":"2024-04-25T17:08:12.161483+0000","last_active":"2024-04-25T17:09:04.145612+0000","last_peered":"2024-04-25T17:09:04.145612+0000","last_clean":"2024-04-25T17:09:04.145612+0000","last_became_active":"2024-04-25T17:08:12.158586+0000","last_became_peered":"2024-04-25T17:08:12.158586+0000","last_unstale":"2024-04-25T17:09:04.145612+0000","last_undegraded":"2024-04-25T17:09:04.145612+0000","last_fullsized":"2024-04-25T17:09:04.145612+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.551328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149766+0000","last_change":"2024-04-25T17:08:11.161497+0000","last_active":"2024-04-25T17:09:04.149766+0000","last_peered":"2024-04-25T17:09:04.149766+0000","last_clean":"2024-04-25T17:09:04.149766+0000","last_became_active":"2024-04-25T17:08:11.160990+0000","last_became_peered":"2024-04-25T17:08:11.160990+0000","last_unstale":"2024-04-25T17:09:04.149766+0000","last_undegraded":"2024-04-25T17:09:04.149766+0000","last_fullsized":"2024-04-25T17:09:04.149766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:59.682133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236920+0000","last_change":"2024-04-25T17:08:12.171214+0000","last_active":"2024-04-25T17:09:04.236920+0000","last_peered":"2024-04-25T17:09:04.236920+0000","last_clean":"2024-04-25T17:09:04.236920+0000","last_became_active":"2024-04-25T17:08:12.170877+0000","last_became_peered":"2024-04-25T17:08:12.170877+0000","last_unstale":"2024-04-25T17:09:04.236920+0000","last_undegraded":"2024-04-25T17:09:04.236920+0000","last_fullsized":"2024-04-25T17:09:04.236920+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:18.442691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145995+0000","last_change":"2024-04-25T17:08:11.154819+0000","last_active":"2024-04-25T17:09:04.145995+0000","last_peered":"2024-04-25T17:09:04.145995+0000","last_clean":"2024-04-25T17:09:04.145995+0000","last_became_active":"2024-04-25T17:08:11.154006+0000","last_became_peered":"2024-04-25T17:08:11.154006+0000","last_unstale":"2024-04-25T17:09:04.145995+0000","last_undegraded":"2024-04-25T17:09:04.145995+0000","last_fullsized":"2024-04-25T17:09:04.145995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.063058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236623+0000","last_change":"2024-04-25T17:08:12.167301+0000","last_active":"2024-04-25T17:09:04.236623+0000","last_peered":"2024-04-25T17:09:04.236623+0000","last_clean":"2024-04-25T17:09:04.236623+0000","last_became_active":"2024-04-25T17:08:12.166944+0000","last_became_peered":"2024-04-25T17:08:12.166944+0000","last_unstale":"2024-04-25T17:09:04.236623+0000","last_undegraded":"2024-04-25T17:09:04.236623+0000","last_fullsized":"2024-04-25T17:09:04.236623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:46.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236606+0000","last_change":"2024-04-25T17:08:11.156616+0000","last_active":"2024-04-25T17:09:04.236606+0000","last_peered":"2024-04-25T17:09:04.236606+0000","last_clean":"2024-04-25T17:09:04.236606+0000","last_became_active":"2024-04-25T17:08:11.156256+0000","last_became_peered":"2024-04-25T17:08:11.156256+0000","last_unstale":"2024-04-25T17:09:04.236606+0000","last_undegraded":"2024-04-25T17:09:04.236606+0000","last_fullsized":"2024-04-25T17:09:04.236606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:37.539773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149432+0000","last_change":"2024-04-25T17:08:12.175216+0000","last_active":"2024-04-25T17:09:04.149432+0000","last_peered":"2024-04-25T17:09:04.149432+0000","last_clean":"2024-04-25T17:09:04.149432+0000","last_became_active":"2024-04-25T17:08:12.174521+0000","last_became_peered":"2024-04-25T17:08:12.174521+0000","last_unstale":"2024-04-25T17:09:04.149432+0000","last_undegraded":"2024-04-25T17:09:04.149432+0000","last_fullsized":"2024-04-25T17:09:04.149432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.322733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236262+0000","last_change":"2024-04-25T17:08:11.162287+0000","last_active":"2024-04-25T17:09:04.236262+0000","last_peered":"2024-04-25T17:09:04.236262+0000","last_clean":"2024-04-25T17:09:04.236262+0000","last_became_active":"2024-04-25T17:08:11.158297+0000","last_became_peered":"2024-04-25T17:08:11.158297+0000","last_unstale":"2024-04-25T17:09:04.236262+0000","last_undegraded":"2024-04-25T17:09:04.236262+0000","last_fullsized":"2024-04-25T17:09:04.236262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:15.758880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147484+0000","last_change":"2024-04-25T17:08:12.163535+0000","last_active":"2024-04-25T17:09:04.147484+0000","last_peered":"2024-04-25T17:09:04.147484+0000","last_clean":"2024-04-25T17:09:04.147484+0000","last_became_active":"2024-04-25T17:08:12.162848+0000","last_became_peered":"2024-04-25T17:08:12.162848+0000","last_unstale":"2024-04-25T17:09:04.147484+0000","last_undegraded":"2024-04-25T17:09:04.147484+0000","last_fullsized":"2024-04-25T17:09:04.147484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:50.288869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.238585+0000","last_change":"2024-04-25T17:09:04.238038+0000","last_active":"2024-04-25T17:09:04.238585+0000","last_peered":"2024-04-25T17:09:04.238585+0000","last_clean":"2024-04-25T17:09:04.238585+0000","last_became_active":"2024-04-25T17:09:04.237635+0000","last_became_peered":"2024-04-25T17:09:04.237635+0000","last_unstale":"2024-04-25T17:09:04.238585+0000","last_undegraded":"2024-04-25T17:09:04.238585+0000","last_fullsized":"2024-04-25T17:09:04.238585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:14.025433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236530+0000","last_change":"2024-04-25T17:08:12.170828+0000","last_active":"2024-04-25T17:09:04.236530+0000","last_peered":"2024-04-25T17:09:04.236530+0000","last_clean":"2024-04-25T17:09:04.236530+0000","last_became_active":"2024-04-25T17:08:12.170479+0000","last_became_peered":"2024-04-25T17:08:12.170479+0000","last_unstale":"2024-04-25T17:09:04.236530+0000","last_undegraded":"2024-04-25T17:09:04.236530+0000","last_fullsized":"2024-04-25T17:09:04.236530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.947118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.157988+0000","last_change":"2024-04-25T17:09:04.157988+0000","last_active":"2024-04-25T17:09:04.157988+0000","last_peered":"2024-04-25T17:09:04.157988+0000","last_clean":"2024-04-25T17:09:04.157988+0000","last_became_active":"2024-04-25T17:09:04.153139+0000","last_became_peered":"2024-04-25T17:09:04.153139+0000","last_unstale":"2024-04-25T17:09:04.157988+0000","last_undegraded":"2024-04-25T17:09:04.157988+0000","last_fullsized":"2024-04-25T17:09:04.157988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:57.969869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150547+0000","last_change":"2024-04-25T17:08:12.162032+0000","last_active":"2024-04-25T17:09:04.150547+0000","last_peered":"2024-04-25T17:09:04.150547+0000","last_clean":"2024-04-25T17:09:04.150547+0000","last_became_active":"2024-04-25T17:08:12.161017+0000","last_became_peered":"2024-04-25T17:08:12.161017+0000","last_unstale":"2024-04-25T17:09:04.150547+0000","last_undegraded":"2024-04-25T17:09:04.150547+0000","last_fullsized":"2024-04-25T17:09:04.150547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.472984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151417+0000","last_change":"2024-04-25T17:08:11.149550+0000","last_active":"2024-04-25T17:09:04.151417+0000","last_peered":"2024-04-25T17:09:04.151417+0000","last_clean":"2024-04-25T17:09:04.151417+0000","last_became_active":"2024-04-25T17:08:11.149082+0000","last_became_peered":"2024-04-25T17:08:11.149082+0000","last_unstale":"2024-04-25T17:09:04.151417+0000","last_undegraded":"2024-04-25T17:09:04.151417+0000","last_fullsized":"2024-04-25T17:09:04.151417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:14.565877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150444+0000","last_change":"2024-04-25T17:08:12.160331+0000","last_active":"2024-04-25T17:09:04.150444+0000","last_peered":"2024-04-25T17:09:04.150444+0000","last_clean":"2024-04-25T17:09:04.150444+0000","last_became_active":"2024-04-25T17:08:12.159931+0000","last_became_peered":"2024-04-25T17:08:12.159931+0000","last_unstale":"2024-04-25T17:09:04.150444+0000","last_undegraded":"2024-04-25T17:09:04.150444+0000","last_fullsized":"2024-04-25T17:09:04.150444+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:37.246974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.146028+0000","last_change":"2024-04-25T17:08:11.163118+0000","last_active":"2024-04-25T17:09:04.146028+0000","last_peered":"2024-04-25T17:09:04.146028+0000","last_clean":"2024-04-25T17:09:04.146028+0000","last_became_active":"2024-04-25T17:08:11.161691+0000","last_became_peered":"2024-04-25T17:08:11.161691+0000","last_unstale":"2024-04-25T17:09:04.146028+0000","last_undegraded":"2024-04-25T17:09:04.146028+0000","last_fullsized":"2024-04-25T17:09:04.146028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:32.006098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149868+0000","last_change":"2024-04-25T17:08:12.168387+0000","last_active":"2024-04-25T17:09:04.149868+0000","last_peered":"2024-04-25T17:09:04.149868+0000","last_clean":"2024-04-25T17:09:04.149868+0000","last_became_active":"2024-04-25T17:08:12.168008+0000","last_became_peered":"2024-04-25T17:08:12.168008+0000","last_unstale":"2024-04-25T17:09:04.149868+0000","last_undegraded":"2024-04-25T17:09:04.149868+0000","last_fullsized":"2024-04-25T17:09:04.149868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:08.438945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145175+0000","last_change":"2024-04-25T17:08:11.149181+0000","last_active":"2024-04-25T17:09:04.145175+0000","last_peered":"2024-04-25T17:09:04.145175+0000","last_clean":"2024-04-25T17:09:04.145175+0000","last_became_active":"2024-04-25T17:08:11.148847+0000","last_became_peered":"2024-04-25T17:08:11.148847+0000","last_unstale":"2024-04-25T17:09:04.145175+0000","last_undegraded":"2024-04-25T17:09:04.145175+0000","last_fullsized":"2024-04-25T17:09:04.145175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:40.720578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150028+0000","last_change":"2024-04-25T17:08:12.162173+0000","last_active":"2024-04-25T17:09:04.150028+0000","last_peered":"2024-04-25T17:09:04.150028+0000","last_clean":"2024-04-25T17:09:04.150028+0000","last_became_active":"2024-04-25T17:08:12.161818+0000","last_became_peered":"2024-04-25T17:08:12.161818+0000","last_unstale":"2024-04-25T17:09:04.150028+0000","last_undegraded":"2024-04-25T17:09:04.150028+0000","last_fullsized":"2024-04-25T17:09:04.150028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:28.872937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147567+0000","last_change":"2024-04-25T17:08:12.165542+0000","last_active":"2024-04-25T17:09:04.147567+0000","last_peered":"2024-04-25T17:09:04.147567+0000","last_clean":"2024-04-25T17:09:04.147567+0000","last_became_active":"2024-04-25T17:08:12.165190+0000","last_became_peered":"2024-04-25T17:08:12.165190+0000","last_unstale":"2024-04-25T17:09:04.147567+0000","last_undegraded":"2024-04-25T17:09:04.147567+0000","last_fullsized":"2024-04-25T17:09:04.147567+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:51.650953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151132+0000","last_change":"2024-04-25T17:08:12.161429+0000","last_active":"2024-04-25T17:09:04.151132+0000","last_peered":"2024-04-25T17:09:04.151132+0000","last_clean":"2024-04-25T17:09:04.151132+0000","last_became_active":"2024-04-25T17:08:12.159184+0000","last_became_peered":"2024-04-25T17:08:12.159184+0000","last_unstale":"2024-04-25T17:09:04.151132+0000","last_undegraded":"2024-04-25T17:09:04.151132+0000","last_fullsized":"2024-04-25T17:09:04.151132+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:56:21.366775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151032+0000","last_change":"2024-04-25T17:08:12.169948+0000","last_active":"2024-04-25T17:09:04.151032+0000","last_peered":"2024-04-25T17:09:04.151032+0000","last_clean":"2024-04-25T17:09:04.151032+0000","last_became_active":"2024-04-25T17:08:12.169592+0000","last_became_peered":"2024-04-25T17:08:12.169592+0000","last_unstale":"2024-04-25T17:09:04.151032+0000","last_undegraded":"2024-04-25T17:09:04.151032+0000","last_fullsized":"2024-04-25T17:09:04.151032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:22.749954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236350+0000","last_change":"2024-04-25T17:08:12.169785+0000","last_active":"2024-04-25T17:09:04.236350+0000","last_peered":"2024-04-25T17:09:04.236350+0000","last_clean":"2024-04-25T17:09:04.236350+0000","last_became_active":"2024-04-25T17:08:12.169226+0000","last_became_peered":"2024-04-25T17:08:12.169226+0000","last_unstale":"2024-04-25T17:09:04.236350+0000","last_undegraded":"2024-04-25T17:09:04.236350+0000","last_fullsized":"2024-04-25T17:09:04.236350+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.232112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145510+0000","last_change":"2024-04-25T17:08:12.157992+0000","last_active":"2024-04-25T17:09:04.145510+0000","last_peered":"2024-04-25T17:09:04.145510+0000","last_clean":"2024-04-25T17:09:04.145510+0000","last_became_active":"2024-04-25T17:08:12.157636+0000","last_became_peered":"2024-04-25T17:08:12.157636+0000","last_unstale":"2024-04-25T17:09:04.145510+0000","last_undegraded":"2024-04-25T17:09:04.145510+0000","last_fullsized":"2024-04-25T17:09:04.145510+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:50.530074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147926+0000","last_change":"2024-04-25T17:08:12.160977+0000","last_active":"2024-04-25T17:09:04.147926+0000","last_peered":"2024-04-25T17:09:04.147926+0000","last_clean":"2024-04-25T17:09:04.147926+0000","last_became_active":"2024-04-25T17:08:12.160571+0000","last_became_peered":"2024-04-25T17:08:12.160571+0000","last_unstale":"2024-04-25T17:09:04.147926+0000","last_undegraded":"2024-04-25T17:09:04.147926+0000","last_fullsized":"2024-04-25T17:09:04.147926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.882628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.153530+0000","last_change":"2024-04-25T17:08:12.158566+0000","last_active":"2024-04-25T17:09:04.153530+0000","last_peered":"2024-04-25T17:09:04.153530+0000","last_clean":"2024-04-25T17:09:04.153530+0000","last_became_active":"2024-04-25T17:08:12.157911+0000","last_became_peered":"2024-04-25T17:08:12.157911+0000","last_unstale":"2024-04-25T17:09:04.153530+0000","last_undegraded":"2024-04-25T17:09:04.153530+0000","last_fullsized":"2024-04-25T17:09:04.153530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:49.665779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236889+0000","last_change":"2024-04-25T17:08:12.174931+0000","last_active":"2024-04-25T17:09:04.236889+0000","last_peered":"2024-04-25T17:09:04.236889+0000","last_clean":"2024-04-25T17:09:04.236889+0000","last_became_active":"2024-04-25T17:08:12.174571+0000","last_became_peered":"2024-04-25T17:08:12.174571+0000","last_unstale":"2024-04-25T17:09:04.236889+0000","last_undegraded":"2024-04-25T17:09:04.236889+0000","last_fullsized":"2024-04-25T17:09:04.236889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:55.832578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145585+0000","last_change":"2024-04-25T17:08:12.165592+0000","last_active":"2024-04-25T17:09:04.145585+0000","last_peered":"2024-04-25T17:09:04.145585+0000","last_clean":"2024-04-25T17:09:04.145585+0000","last_became_active":"2024-04-25T17:08:12.165273+0000","last_became_peered":"2024-04-25T17:08:12.165273+0000","last_unstale":"2024-04-25T17:09:04.145585+0000","last_undegraded":"2024-04-25T17:09:04.145585+0000","last_fullsized":"2024-04-25T17:09:04.145585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:25.199657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236235+0000","last_change":"2024-04-25T17:08:12.174095+0000","last_active":"2024-04-25T17:09:04.236235+0000","last_peered":"2024-04-25T17:09:04.236235+0000","last_clean":"2024-04-25T17:09:04.236235+0000","last_became_active":"2024-04-25T17:08:12.172786+0000","last_became_peered":"2024-04-25T17:08:12.172786+0000","last_unstale":"2024-04-25T17:09:04.236235+0000","last_undegraded":"2024-04-25T17:09:04.236235+0000","last_fullsized":"2024-04-25T17:09:04.236235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.736300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236639+0000","last_change":"2024-04-25T17:08:12.158772+0000","last_active":"2024-04-25T17:09:04.236639+0000","last_peered":"2024-04-25T17:09:04.236639+0000","last_clean":"2024-04-25T17:09:04.236639+0000","last_became_active":"2024-04-25T17:08:12.158417+0000","last_became_peered":"2024-04-25T17:08:12.158417+0000","last_unstale":"2024-04-25T17:09:04.236639+0000","last_undegraded":"2024-04-25T17:09:04.236639+0000","last_fullsized":"2024-04-25T17:09:04.236639+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:21.837887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145472+0000","last_change":"2024-04-25T17:08:12.162384+0000","last_active":"2024-04-25T17:09:04.145472+0000","last_peered":"2024-04-25T17:09:04.145472+0000","last_clean":"2024-04-25T17:09:04.145472+0000","last_became_active":"2024-04-25T17:08:12.161997+0000","last_became_peered":"2024-04-25T17:08:12.161997+0000","last_unstale":"2024-04-25T17:09:04.145472+0000","last_undegraded":"2024-04-25T17:09:04.145472+0000","last_fullsized":"2024-04-25T17:09:04.145472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:17.358171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147625+0000","last_change":"2024-04-25T17:08:12.159015+0000","last_active":"2024-04-25T17:09:04.147625+0000","last_peered":"2024-04-25T17:09:04.147625+0000","last_clean":"2024-04-25T17:09:04.147625+0000","last_became_active":"2024-04-25T17:08:12.157999+0000","last_became_peered":"2024-04-25T17:08:12.157999+0000","last_unstale":"2024-04-25T17:09:04.147625+0000","last_undegraded":"2024-04-25T17:09:04.147625+0000","last_fullsized":"2024-04-25T17:09:04.147625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:00.291221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280965,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61844,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":161,"num_read_kb":142,"num_write":75,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":185532,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678044,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":835584,"data_stored":406301,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874274,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1058026,"data_compressed":43374,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037800,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":406883,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":406323,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430259,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":997368,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":406418,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":467929,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":839680,"data_stored":406391,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":467381,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":406639,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986242,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":406323,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822717,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":996968,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61628,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61080,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61240,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T17:09:32.950 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T17:09:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:33 smithi096 ceph-mon[29131]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:09:33.216 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:33.397 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:33 smithi097 ceph-mon[32602]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:09:33.451 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:33 smithi171 ceph-mon[33189]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:09:34.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:34 smithi096 ceph-mon[29131]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:34.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:34 smithi097 ceph-mon[32602]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:34.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:34 smithi171 ceph-mon[33189]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 -- 172.21.15.96:0/2508086349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948103a70 msgr2=0x7fb948107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 --2- 172.21.15.96:0/2508086349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948103a70 0x7fb948107fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009990 tx=0x7fb93c009ca0 comp rx=0 tx=0).stop 2024-04-25T17:09:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 -- 172.21.15.96:0/2508086349 shutdown_connections 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 --2- 172.21.15.96:0/2508086349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb948108570 0x7fb948108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 --2- 172.21.15.96:0/2508086349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb948103120 0x7fb948103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 --2- 172.21.15.96:0/2508086349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948103a70 0x7fb948107fa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 -- 172.21.15.96:0/2508086349 >> 172.21.15.96:0/2508086349 conn(0x7fb9480fec90 msgr2=0x7fb9481010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 -- 172.21.15.96:0/2508086349 shutdown_connections 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.895+0000 7fb94e27f700 1 -- 172.21.15.96:0/2508086349 wait complete. 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 Processor -- start 2024-04-25T17:09:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 -- start start 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb948103120 0x7fb9481a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb948103a70 0x7fb9481a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948108570 0x7fb9481a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb94811a900 con 0x7fb948108570 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb94811a780 con 0x7fb948103120 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94e27f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb94811aa80 con 0x7fb948103a70 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94c81c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948108570 0x7fb9481a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94c81c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948108570 0x7fb9481a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58912/0 (socket says 172.21.15.96:58912) 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb94c81c700 1 -- 172.21.15.96:0/1636051162 learned_addr learned my addr 172.21.15.96:0/1636051162 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.896+0000 7fb9477fe700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb948103a70 0x7fb9481a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:34.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb947fff700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb948103120 0x7fb9481a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94c81c700 1 -- 172.21.15.96:0/1636051162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb948103a70 msgr2=0x7fb9481a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94c81c700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb948103a70 0x7fb9481a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94c81c700 1 -- 172.21.15.96:0/1636051162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb948103120 msgr2=0x7fb9481a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94c81c700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb948103120 0x7fb9481a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94c81c700 1 -- 172.21.15.96:0/1636051162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9481a6130 con 0x7fb948108570 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94c81c700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948108570 0x7fb9481a5b40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb940007d70 tx=0x7fb94000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb9457fa700 1 -- 172.21.15.96:0/1636051162 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940016020 con 0x7fb948108570 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9481a6420 con 0x7fb948108570 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb9457fa700 1 -- 172.21.15.96:0/1636051162 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb94000ad40 con 0x7fb948108570 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb9457fa700 1 -- 172.21.15.96:0/1636051162 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9400115e0 con 0x7fb948108570 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.897+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481a6930 con 0x7fb948108570 2024-04-25T17:09:34.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.898+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948051190 con 0x7fb948108570 2024-04-25T17:09:34.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.899+0000 7fb9457fa700 1 -- 172.21.15.96:0/1636051162 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb940019020 con 0x7fb948108570 2024-04-25T17:09:34.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.900+0000 7fb9457fa700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb9300afb80 0x7fb9300b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:34.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.900+0000 7fb947fff700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb9300afb80 0x7fb9300b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:34.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.901+0000 7fb9457fa700 1 -- 172.21.15.96:0/1636051162 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fb9400df120 con 0x7fb948108570 2024-04-25T17:09:34.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.901+0000 7fb947fff700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb9300afb80 0x7fb9300b2090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb938005fd0 tx=0x7fb93800a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:34.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:34.904+0000 7fb9457fa700 1 -- 172.21.15.96:0/1636051162 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb940020020 con 0x7fb948108570 2024-04-25T17:09:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.099+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9481075f0 con 0x7fb9300afb80 2024-04-25T17:09:35.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.106+0000 7fb9457fa700 1 -- 172.21.15.96:0/1636051162 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259052 (secure 0 0 0) 0x7fb9481075f0 con 0x7fb9300afb80 2024-04-25T17:09:35.109 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb9300afb80 msgr2=0x7fb9300b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb9300afb80 0x7fb9300b2090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb938005fd0 tx=0x7fb93800a560 comp rx=0 tx=0).stop 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948108570 msgr2=0x7fb9481a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948108570 0x7fb9481a5b40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb940007d70 tx=0x7fb94000a340 comp rx=0 tx=0).stop 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 shutdown_connections 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb9300afb80 0x7fb9300b2090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb948108570 0x7fb9481a5b40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb948103120 0x7fb9481a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 --2- 172.21.15.96:0/1636051162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb948103a70 0x7fb9481a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 >> 172.21.15.96:0/1636051162 conn(0x7fb9480fec90 msgr2=0x7fb948105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 shutdown_connections 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:35.109+0000 7fb94e27f700 1 -- 172.21.15.96:0/1636051162 wait complete. 2024-04-25T17:09:35.111 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T17:09:35.413 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:35 smithi096 ceph-mon[29131]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T17:09:35.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:35 smithi097 ceph-mon[32602]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T17:09:35.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:35 smithi171 ceph-mon[33189]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T17:09:35.741 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-25T17:09:35.083010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652695,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":253,"num_read_kb":217,"num_write":203,"num_write_kb":2238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2952422,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":377,"ondisk_log_size":377,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359424,"kb_used_data":10212,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582400,"statfs":{"total":1151940427776,"available":1151572377600,"internally_reserved":0,"allocated":10457088,"data_stored":6833199,"data_compressed":182331,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.148182+0000","last_change":"2024-04-25T17:08:12.162430+0000","last_active":"2024-04-25T17:09:04.148182+0000","last_peered":"2024-04-25T17:09:04.148182+0000","last_clean":"2024-04-25T17:09:04.148182+0000","last_became_active":"2024-04-25T17:08:12.162052+0000","last_became_peered":"2024-04-25T17:08:12.162052+0000","last_unstale":"2024-04-25T17:09:04.148182+0000","last_undegraded":"2024-04-25T17:09:04.148182+0000","last_fullsized":"2024-04-25T17:09:04.148182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:09.637937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.075552+0000","last_change":"2024-04-25T17:08:12.169139+0000","last_active":"2024-04-25T17:09:33.075552+0000","last_peered":"2024-04-25T17:09:33.075552+0000","last_clean":"2024-04-25T17:09:33.075552+0000","last_became_active":"2024-04-25T17:08:12.168715+0000","last_became_peered":"2024-04-25T17:08:12.168715+0000","last_unstale":"2024-04-25T17:09:33.075552+0000","last_undegraded":"2024-04-25T17:09:33.075552+0000","last_fullsized":"2024-04-25T17:09:33.075552+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:57.357750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.088092+0000","last_change":"2024-04-25T17:08:12.159141+0000","last_active":"2024-04-25T17:09:33.088092+0000","last_peered":"2024-04-25T17:09:33.088092+0000","last_clean":"2024-04-25T17:09:33.088092+0000","last_became_active":"2024-04-25T17:08:12.158796+0000","last_became_peered":"2024-04-25T17:08:12.158796+0000","last_unstale":"2024-04-25T17:09:33.088092+0000","last_undegraded":"2024-04-25T17:09:33.088092+0000","last_fullsized":"2024-04-25T17:09:33.088092+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:55.635629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145879+0000","last_change":"2024-04-25T17:08:12.160464+0000","last_active":"2024-04-25T17:09:04.145879+0000","last_peered":"2024-04-25T17:09:04.145879+0000","last_clean":"2024-04-25T17:09:04.145879+0000","last_became_active":"2024-04-25T17:08:12.160118+0000","last_became_peered":"2024-04-25T17:08:12.160118+0000","last_unstale":"2024-04-25T17:09:04.145879+0000","last_undegraded":"2024-04-25T17:09:04.145879+0000","last_fullsized":"2024-04-25T17:09:04.145879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:59:06.465964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149631+0000","last_change":"2024-04-25T17:08:12.174591+0000","last_active":"2024-04-25T17:09:04.149631+0000","last_peered":"2024-04-25T17:09:04.149631+0000","last_clean":"2024-04-25T17:09:04.149631+0000","last_became_active":"2024-04-25T17:08:12.174173+0000","last_became_peered":"2024-04-25T17:08:12.174173+0000","last_unstale":"2024-04-25T17:09:04.149631+0000","last_undegraded":"2024-04-25T17:09:04.149631+0000","last_fullsized":"2024-04-25T17:09:04.149631+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:12.028658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.078406+0000","last_change":"2024-04-25T17:08:12.164230+0000","last_active":"2024-04-25T17:09:33.078406+0000","last_peered":"2024-04-25T17:09:33.078406+0000","last_clean":"2024-04-25T17:09:33.078406+0000","last_became_active":"2024-04-25T17:08:12.163868+0000","last_became_peered":"2024-04-25T17:08:12.163868+0000","last_unstale":"2024-04-25T17:09:33.078406+0000","last_undegraded":"2024-04-25T17:09:33.078406+0000","last_fullsized":"2024-04-25T17:09:33.078406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:12.604164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149993+0000","last_change":"2024-04-25T17:08:12.161627+0000","last_active":"2024-04-25T17:09:04.149993+0000","last_peered":"2024-04-25T17:09:04.149993+0000","last_clean":"2024-04-25T17:09:04.149993+0000","last_became_active":"2024-04-25T17:08:12.161146+0000","last_became_peered":"2024-04-25T17:08:12.161146+0000","last_unstale":"2024-04-25T17:09:04.149993+0000","last_undegraded":"2024-04-25T17:09:04.149993+0000","last_fullsized":"2024-04-25T17:09:04.149993+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.972053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236010+0000","last_change":"2024-04-25T17:08:12.165512+0000","last_active":"2024-04-25T17:09:04.236010+0000","last_peered":"2024-04-25T17:09:04.236010+0000","last_clean":"2024-04-25T17:09:04.236010+0000","last_became_active":"2024-04-25T17:08:12.164018+0000","last_became_peered":"2024-04-25T17:08:12.164018+0000","last_unstale":"2024-04-25T17:09:04.236010+0000","last_undegraded":"2024-04-25T17:09:04.236010+0000","last_fullsized":"2024-04-25T17:09:04.236010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:28.075786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.086451+0000","last_change":"2024-04-25T17:08:12.173313+0000","last_active":"2024-04-25T17:09:33.086451+0000","last_peered":"2024-04-25T17:09:33.086451+0000","last_clean":"2024-04-25T17:09:33.086451+0000","last_became_active":"2024-04-25T17:08:12.171269+0000","last_became_peered":"2024-04-25T17:08:12.171269+0000","last_unstale":"2024-04-25T17:09:33.086451+0000","last_undegraded":"2024-04-25T17:09:33.086451+0000","last_fullsized":"2024-04-25T17:09:33.086451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:39.565973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"78'16","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.077769+0000","last_change":"2024-04-25T17:09:04.156007+0000","last_active":"2024-04-25T17:09:33.077769+0000","last_peered":"2024-04-25T17:09:33.077769+0000","last_clean":"2024-04-25T17:09:33.077769+0000","last_became_active":"2024-04-25T17:09:04.155636+0000","last_became_peered":"2024-04-25T17:09:04.155636+0000","last_unstale":"2024-04-25T17:09:33.077769+0000","last_undegraded":"2024-04-25T17:09:33.077769+0000","last_fullsized":"2024-04-25T17:09:33.077769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:30.993817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150301+0000","last_change":"2024-04-25T17:08:12.160435+0000","last_active":"2024-04-25T17:09:04.150301+0000","last_peered":"2024-04-25T17:09:04.150301+0000","last_clean":"2024-04-25T17:09:04.150301+0000","last_became_active":"2024-04-25T17:08:12.160085+0000","last_became_peered":"2024-04-25T17:08:12.160085+0000","last_unstale":"2024-04-25T17:09:04.150301+0000","last_undegraded":"2024-04-25T17:09:04.150301+0000","last_fullsized":"2024-04-25T17:09:04.150301+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:16.709567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145796+0000","last_change":"2024-04-25T17:08:12.173990+0000","last_active":"2024-04-25T17:09:04.145796+0000","last_peered":"2024-04-25T17:09:04.145796+0000","last_clean":"2024-04-25T17:09:04.145796+0000","last_became_active":"2024-04-25T17:08:12.172838+0000","last_became_peered":"2024-04-25T17:08:12.172838+0000","last_unstale":"2024-04-25T17:09:04.145796+0000","last_undegraded":"2024-04-25T17:09:04.145796+0000","last_fullsized":"2024-04-25T17:09:04.145796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:00:56.720758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.083296+0000","last_change":"2024-04-25T17:08:12.167579+0000","last_active":"2024-04-25T17:09:33.083296+0000","last_peered":"2024-04-25T17:09:33.083296+0000","last_clean":"2024-04-25T17:09:33.083296+0000","last_became_active":"2024-04-25T17:08:12.166583+0000","last_became_peered":"2024-04-25T17:08:12.166583+0000","last_unstale":"2024-04-25T17:09:33.083296+0000","last_undegraded":"2024-04-25T17:09:33.083296+0000","last_fullsized":"2024-04-25T17:09:33.083296+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:44.081814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.078064+0000","last_change":"2024-04-25T17:08:12.161263+0000","last_active":"2024-04-25T17:09:33.078064+0000","last_peered":"2024-04-25T17:09:33.078064+0000","last_clean":"2024-04-25T17:09:33.078064+0000","last_became_active":"2024-04-25T17:08:12.160078+0000","last_became_peered":"2024-04-25T17:08:12.160078+0000","last_unstale":"2024-04-25T17:09:33.078064+0000","last_undegraded":"2024-04-25T17:09:33.078064+0000","last_fullsized":"2024-04-25T17:09:33.078064+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:56.107641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.076153+0000","last_change":"2024-04-25T17:08:12.167318+0000","last_active":"2024-04-25T17:09:33.076153+0000","last_peered":"2024-04-25T17:09:33.076153+0000","last_clean":"2024-04-25T17:09:33.076153+0000","last_became_active":"2024-04-25T17:08:12.166964+0000","last_became_peered":"2024-04-25T17:08:12.166964+0000","last_unstale":"2024-04-25T17:09:33.076153+0000","last_undegraded":"2024-04-25T17:09:33.076153+0000","last_fullsized":"2024-04-25T17:09:33.076153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:50:25.967818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.090754+0000","last_change":"2024-04-25T17:08:12.166417+0000","last_active":"2024-04-25T17:09:33.090754+0000","last_peered":"2024-04-25T17:09:33.090754+0000","last_clean":"2024-04-25T17:09:33.090754+0000","last_became_active":"2024-04-25T17:08:12.165928+0000","last_became_peered":"2024-04-25T17:08:12.165928+0000","last_unstale":"2024-04-25T17:09:33.090754+0000","last_undegraded":"2024-04-25T17:09:33.090754+0000","last_fullsized":"2024-04-25T17:09:33.090754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:24:22.359814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.148262+0000","last_change":"2024-04-25T17:08:12.164952+0000","last_active":"2024-04-25T17:09:04.148262+0000","last_peered":"2024-04-25T17:09:04.148262+0000","last_clean":"2024-04-25T17:09:04.148262+0000","last_became_active":"2024-04-25T17:08:12.164595+0000","last_became_peered":"2024-04-25T17:08:12.164595+0000","last_unstale":"2024-04-25T17:09:04.148262+0000","last_undegraded":"2024-04-25T17:09:04.148262+0000","last_fullsized":"2024-04-25T17:09:04.148262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:45.020512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.066973+0000","last_change":"2024-04-25T17:08:12.153583+0000","last_active":"2024-04-25T17:09:33.066973+0000","last_peered":"2024-04-25T17:09:33.066973+0000","last_clean":"2024-04-25T17:09:33.066973+0000","last_became_active":"2024-04-25T17:08:12.153070+0000","last_became_peered":"2024-04-25T17:08:12.153070+0000","last_unstale":"2024-04-25T17:09:33.066973+0000","last_undegraded":"2024-04-25T17:09:33.066973+0000","last_fullsized":"2024-04-25T17:09:33.066973+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:34.151002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151671+0000","last_change":"2024-04-25T17:08:12.166837+0000","last_active":"2024-04-25T17:09:04.151671+0000","last_peered":"2024-04-25T17:09:04.151671+0000","last_clean":"2024-04-25T17:09:04.151671+0000","last_became_active":"2024-04-25T17:08:12.166477+0000","last_became_peered":"2024-04-25T17:08:12.166477+0000","last_unstale":"2024-04-25T17:09:04.151671+0000","last_undegraded":"2024-04-25T17:09:04.151671+0000","last_fullsized":"2024-04-25T17:09:04.151671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:50.492333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.241787+0000","last_change":"2024-04-25T17:09:04.241787+0000","last_active":"2024-04-25T17:09:04.241787+0000","last_peered":"2024-04-25T17:09:04.241787+0000","last_clean":"2024-04-25T17:09:04.241787+0000","last_became_active":"2024-04-25T17:09:04.237983+0000","last_became_peered":"2024-04-25T17:09:04.237983+0000","last_unstale":"2024-04-25T17:09:04.241787+0000","last_undegraded":"2024-04-25T17:09:04.241787+0000","last_fullsized":"2024-04-25T17:09:04.241787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:02.752902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236906+0000","last_change":"2024-04-25T17:08:12.178117+0000","last_active":"2024-04-25T17:09:04.236906+0000","last_peered":"2024-04-25T17:09:04.236906+0000","last_clean":"2024-04-25T17:09:04.236906+0000","last_became_active":"2024-04-25T17:08:12.177767+0000","last_became_peered":"2024-04-25T17:08:12.177767+0000","last_unstale":"2024-04-25T17:09:04.236906+0000","last_undegraded":"2024-04-25T17:09:04.236906+0000","last_fullsized":"2024-04-25T17:09:04.236906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:34.488042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.237070+0000","last_change":"2024-04-25T17:08:11.153996+0000","last_active":"2024-04-25T17:09:04.237070+0000","last_peered":"2024-04-25T17:09:04.237070+0000","last_clean":"2024-04-25T17:09:04.237070+0000","last_became_active":"2024-04-25T17:08:11.152993+0000","last_became_peered":"2024-04-25T17:08:11.152993+0000","last_unstale":"2024-04-25T17:09:04.237070+0000","last_undegraded":"2024-04-25T17:09:04.237070+0000","last_fullsized":"2024-04-25T17:09:04.237070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:33.116616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"78'6","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.087755+0000","last_change":"2024-04-25T17:08:12.163707+0000","last_active":"2024-04-25T17:09:33.087755+0000","last_peered":"2024-04-25T17:09:33.087755+0000","last_clean":"2024-04-25T17:09:33.087755+0000","last_became_active":"2024-04-25T17:08:12.161868+0000","last_became_peered":"2024-04-25T17:08:12.161868+0000","last_unstale":"2024-04-25T17:09:33.087755+0000","last_undegraded":"2024-04-25T17:09:33.087755+0000","last_fullsized":"2024-04-25T17:09:33.087755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.333036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.160023+0000","last_change":"2024-04-25T17:09:04.160023+0000","last_active":"2024-04-25T17:09:04.160023+0000","last_peered":"2024-04-25T17:09:04.160023+0000","last_clean":"2024-04-25T17:09:04.160023+0000","last_became_active":"2024-04-25T17:09:04.154073+0000","last_became_peered":"2024-04-25T17:09:04.154073+0000","last_unstale":"2024-04-25T17:09:04.160023+0000","last_undegraded":"2024-04-25T17:09:04.160023+0000","last_fullsized":"2024-04-25T17:09:04.160023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:59.181852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.079950+0000","last_change":"2024-04-25T17:08:12.170278+0000","last_active":"2024-04-25T17:09:33.079950+0000","last_peered":"2024-04-25T17:09:33.079950+0000","last_clean":"2024-04-25T17:09:33.079950+0000","last_became_active":"2024-04-25T17:08:12.169219+0000","last_became_peered":"2024-04-25T17:08:12.169219+0000","last_unstale":"2024-04-25T17:09:33.079950+0000","last_undegraded":"2024-04-25T17:09:33.079950+0000","last_fullsized":"2024-04-25T17:09:33.079950+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:35.551549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149829+0000","last_change":"2024-04-25T17:08:11.153167+0000","last_active":"2024-04-25T17:09:04.149829+0000","last_peered":"2024-04-25T17:09:04.149829+0000","last_clean":"2024-04-25T17:09:04.149829+0000","last_became_active":"2024-04-25T17:08:11.152772+0000","last_became_peered":"2024-04-25T17:08:11.152772+0000","last_unstale":"2024-04-25T17:09:04.149829+0000","last_undegraded":"2024-04-25T17:09:04.149829+0000","last_fullsized":"2024-04-25T17:09:04.149829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:57.198963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.146054+0000","last_change":"2024-04-25T17:08:12.163530+0000","last_active":"2024-04-25T17:09:04.146054+0000","last_peered":"2024-04-25T17:09:04.146054+0000","last_clean":"2024-04-25T17:09:04.146054+0000","last_became_active":"2024-04-25T17:08:12.163179+0000","last_became_peered":"2024-04-25T17:08:12.163179+0000","last_unstale":"2024-04-25T17:09:04.146054+0000","last_undegraded":"2024-04-25T17:09:04.146054+0000","last_fullsized":"2024-04-25T17:09:04.146054+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:59.536274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.155403+0000","last_change":"2024-04-25T17:09:04.154296+0000","last_active":"2024-04-25T17:09:04.155403+0000","last_peered":"2024-04-25T17:09:04.155403+0000","last_clean":"2024-04-25T17:09:04.155403+0000","last_became_active":"2024-04-25T17:09:04.153661+0000","last_became_peered":"2024-04-25T17:09:04.153661+0000","last_unstale":"2024-04-25T17:09:04.155403+0000","last_undegraded":"2024-04-25T17:09:04.155403+0000","last_fullsized":"2024-04-25T17:09:04.155403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:44.170563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149797+0000","last_change":"2024-04-25T17:08:12.167584+0000","last_active":"2024-04-25T17:09:04.149797+0000","last_peered":"2024-04-25T17:09:04.149797+0000","last_clean":"2024-04-25T17:09:04.149797+0000","last_became_active":"2024-04-25T17:08:12.166717+0000","last_became_peered":"2024-04-25T17:08:12.166717+0000","last_unstale":"2024-04-25T17:09:04.149797+0000","last_undegraded":"2024-04-25T17:09:04.149797+0000","last_fullsized":"2024-04-25T17:09:04.149797+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:40:38.047671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149330+0000","last_change":"2024-04-25T17:09:04.149330+0000","last_active":"2024-04-25T17:09:04.149330+0000","last_peered":"2024-04-25T17:09:04.149330+0000","last_clean":"2024-04-25T17:09:04.149330+0000","last_became_active":"2024-04-25T17:09:04.148884+0000","last_became_peered":"2024-04-25T17:09:04.148884+0000","last_unstale":"2024-04-25T17:09:04.149330+0000","last_undegraded":"2024-04-25T17:09:04.149330+0000","last_fullsized":"2024-04-25T17:09:04.149330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:25.787948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145804+0000","last_change":"2024-04-25T17:08:12.170409+0000","last_active":"2024-04-25T17:09:04.145804+0000","last_peered":"2024-04-25T17:09:04.145804+0000","last_clean":"2024-04-25T17:09:04.145804+0000","last_became_active":"2024-04-25T17:08:12.169510+0000","last_became_peered":"2024-04-25T17:08:12.169510+0000","last_unstale":"2024-04-25T17:09:04.145804+0000","last_undegraded":"2024-04-25T17:09:04.145804+0000","last_fullsized":"2024-04-25T17:09:04.145804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:19.310486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.144829+0000","last_change":"2024-04-25T17:08:11.150980+0000","last_active":"2024-04-25T17:09:04.144829+0000","last_peered":"2024-04-25T17:09:04.144829+0000","last_clean":"2024-04-25T17:09:04.144829+0000","last_became_active":"2024-04-25T17:08:11.150580+0000","last_became_peered":"2024-04-25T17:08:11.150580+0000","last_unstale":"2024-04-25T17:09:04.144829+0000","last_undegraded":"2024-04-25T17:09:04.144829+0000","last_fullsized":"2024-04-25T17:09:04.144829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:59.539673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236048+0000","last_change":"2024-04-25T17:08:12.171013+0000","last_active":"2024-04-25T17:09:04.236048+0000","last_peered":"2024-04-25T17:09:04.236048+0000","last_clean":"2024-04-25T17:09:04.236048+0000","last_became_active":"2024-04-25T17:08:12.169868+0000","last_became_peered":"2024-04-25T17:08:12.169868+0000","last_unstale":"2024-04-25T17:09:04.236048+0000","last_undegraded":"2024-04-25T17:09:04.236048+0000","last_fullsized":"2024-04-25T17:09:04.236048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:22.418990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145580+0000","last_change":"2024-04-25T17:08:11.157618+0000","last_active":"2024-04-25T17:09:04.145580+0000","last_peered":"2024-04-25T17:09:04.145580+0000","last_clean":"2024-04-25T17:09:04.145580+0000","last_became_active":"2024-04-25T17:08:11.157052+0000","last_became_peered":"2024-04-25T17:08:11.157052+0000","last_unstale":"2024-04-25T17:09:04.145580+0000","last_undegraded":"2024-04-25T17:09:04.145580+0000","last_fullsized":"2024-04-25T17:09:04.145580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:06.032163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.062755+0000","last_change":"2024-04-25T17:08:12.151093+0000","last_active":"2024-04-25T17:09:33.062755+0000","last_peered":"2024-04-25T17:09:33.062755+0000","last_clean":"2024-04-25T17:09:33.062755+0000","last_became_active":"2024-04-25T17:08:12.150699+0000","last_became_peered":"2024-04-25T17:08:12.150699+0000","last_unstale":"2024-04-25T17:09:33.062755+0000","last_undegraded":"2024-04-25T17:09:33.062755+0000","last_fullsized":"2024-04-25T17:09:33.062755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:36:27.892652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.148011+0000","last_change":"2024-04-25T17:08:11.157673+0000","last_active":"2024-04-25T17:09:04.148011+0000","last_peered":"2024-04-25T17:09:04.148011+0000","last_clean":"2024-04-25T17:09:04.148011+0000","last_became_active":"2024-04-25T17:08:11.157089+0000","last_became_peered":"2024-04-25T17:08:11.157089+0000","last_unstale":"2024-04-25T17:09:04.148011+0000","last_undegraded":"2024-04-25T17:09:04.148011+0000","last_fullsized":"2024-04-25T17:09:04.148011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:47.586003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.074826+0000","last_change":"2024-04-25T17:08:12.170666+0000","last_active":"2024-04-25T17:09:33.074826+0000","last_peered":"2024-04-25T17:09:33.074826+0000","last_clean":"2024-04-25T17:09:33.074826+0000","last_became_active":"2024-04-25T17:08:12.169607+0000","last_became_peered":"2024-04-25T17:08:12.169607+0000","last_unstale":"2024-04-25T17:09:33.074826+0000","last_undegraded":"2024-04-25T17:09:33.074826+0000","last_fullsized":"2024-04-25T17:09:33.074826+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:34.624439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149916+0000","last_change":"2024-04-25T17:08:11.153407+0000","last_active":"2024-04-25T17:09:04.149916+0000","last_peered":"2024-04-25T17:09:04.149916+0000","last_clean":"2024-04-25T17:09:04.149916+0000","last_became_active":"2024-04-25T17:08:11.152932+0000","last_became_peered":"2024-04-25T17:08:11.152932+0000","last_unstale":"2024-04-25T17:09:04.149916+0000","last_undegraded":"2024-04-25T17:09:04.149916+0000","last_fullsized":"2024-04-25T17:09:04.149916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:06.712620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.078177+0000","last_change":"2024-04-25T17:08:12.173788+0000","last_active":"2024-04-25T17:09:33.078177+0000","last_peered":"2024-04-25T17:09:33.078177+0000","last_clean":"2024-04-25T17:09:33.078177+0000","last_became_active":"2024-04-25T17:08:12.172714+0000","last_became_peered":"2024-04-25T17:08:12.172714+0000","last_unstale":"2024-04-25T17:09:33.078177+0000","last_undegraded":"2024-04-25T17:09:33.078177+0000","last_fullsized":"2024-04-25T17:09:33.078177+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:18.892187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.146168+0000","last_change":"2024-04-25T17:08:11.159802+0000","last_active":"2024-04-25T17:09:04.146168+0000","last_peered":"2024-04-25T17:09:04.146168+0000","last_clean":"2024-04-25T17:09:04.146168+0000","last_became_active":"2024-04-25T17:08:11.159431+0000","last_became_peered":"2024-04-25T17:08:11.159431+0000","last_unstale":"2024-04-25T17:09:04.146168+0000","last_undegraded":"2024-04-25T17:09:04.146168+0000","last_fullsized":"2024-04-25T17:09:04.146168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.557183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.089003+0000","last_change":"2024-04-25T17:08:12.162703+0000","last_active":"2024-04-25T17:09:33.089003+0000","last_peered":"2024-04-25T17:09:33.089003+0000","last_clean":"2024-04-25T17:09:33.089003+0000","last_became_active":"2024-04-25T17:08:12.162361+0000","last_became_peered":"2024-04-25T17:08:12.162361+0000","last_unstale":"2024-04-25T17:09:33.089003+0000","last_undegraded":"2024-04-25T17:09:33.089003+0000","last_fullsized":"2024-04-25T17:09:33.089003+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:51.262319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'29","reported_seq":214,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.197093+0000","last_change":"2024-04-25T17:08:11.153938+0000","last_active":"2024-04-25T17:09:33.197093+0000","last_peered":"2024-04-25T17:09:33.197093+0000","last_clean":"2024-04-25T17:09:33.197093+0000","last_became_active":"2024-04-25T17:08:11.153591+0000","last_became_peered":"2024-04-25T17:08:11.153591+0000","last_unstale":"2024-04-25T17:09:33.197093+0000","last_undegraded":"2024-04-25T17:09:33.197093+0000","last_fullsized":"2024-04-25T17:09:33.197093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.802333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":166,"num_read_kb":142,"num_write":29,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149884+0000","last_change":"2024-04-25T17:08:12.156045+0000","last_active":"2024-04-25T17:09:04.149884+0000","last_peered":"2024-04-25T17:09:04.149884+0000","last_clean":"2024-04-25T17:09:04.149884+0000","last_became_active":"2024-04-25T17:08:12.155657+0000","last_became_peered":"2024-04-25T17:08:12.155657+0000","last_unstale":"2024-04-25T17:09:04.149884+0000","last_undegraded":"2024-04-25T17:09:04.149884+0000","last_fullsized":"2024-04-25T17:09:04.149884+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:17.757269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151133+0000","last_change":"2024-04-25T17:08:11.162009+0000","last_active":"2024-04-25T17:09:04.151133+0000","last_peered":"2024-04-25T17:09:04.151133+0000","last_clean":"2024-04-25T17:09:04.151133+0000","last_became_active":"2024-04-25T17:08:11.161574+0000","last_became_peered":"2024-04-25T17:08:11.161574+0000","last_unstale":"2024-04-25T17:09:04.151133+0000","last_undegraded":"2024-04-25T17:09:04.151133+0000","last_fullsized":"2024-04-25T17:09:04.151133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:16.760355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.080540+0000","last_change":"2024-04-25T17:08:12.159522+0000","last_active":"2024-04-25T17:09:33.080540+0000","last_peered":"2024-04-25T17:09:33.080540+0000","last_clean":"2024-04-25T17:09:33.080540+0000","last_became_active":"2024-04-25T17:08:12.159036+0000","last_became_peered":"2024-04-25T17:08:12.159036+0000","last_unstale":"2024-04-25T17:09:33.080540+0000","last_undegraded":"2024-04-25T17:09:33.080540+0000","last_fullsized":"2024-04-25T17:09:33.080540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:32.044488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145694+0000","last_change":"2024-04-25T17:08:11.162487+0000","last_active":"2024-04-25T17:09:04.145694+0000","last_peered":"2024-04-25T17:09:04.145694+0000","last_clean":"2024-04-25T17:09:04.145694+0000","last_became_active":"2024-04-25T17:08:11.161961+0000","last_became_peered":"2024-04-25T17:08:11.161961+0000","last_unstale":"2024-04-25T17:09:04.145694+0000","last_undegraded":"2024-04-25T17:09:04.145694+0000","last_fullsized":"2024-04-25T17:09:04.145694+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:02.643683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150472+0000","last_change":"2024-04-25T17:08:12.164107+0000","last_active":"2024-04-25T17:09:04.150472+0000","last_peered":"2024-04-25T17:09:04.150472+0000","last_clean":"2024-04-25T17:09:04.150472+0000","last_became_active":"2024-04-25T17:08:12.163741+0000","last_became_peered":"2024-04-25T17:08:12.163741+0000","last_unstale":"2024-04-25T17:09:04.150472+0000","last_undegraded":"2024-04-25T17:09:04.150472+0000","last_fullsized":"2024-04-25T17:09:04.150472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:00.552833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145227+0000","last_change":"2024-04-25T17:08:11.153383+0000","last_active":"2024-04-25T17:09:04.145227+0000","last_peered":"2024-04-25T17:09:04.145227+0000","last_clean":"2024-04-25T17:09:04.145227+0000","last_became_active":"2024-04-25T17:08:11.152573+0000","last_became_peered":"2024-04-25T17:08:11.152573+0000","last_unstale":"2024-04-25T17:09:04.145227+0000","last_undegraded":"2024-04-25T17:09:04.145227+0000","last_fullsized":"2024-04-25T17:09:04.145227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:13.006626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149724+0000","last_change":"2024-04-25T17:08:12.165939+0000","last_active":"2024-04-25T17:09:04.149724+0000","last_peered":"2024-04-25T17:09:04.149724+0000","last_clean":"2024-04-25T17:09:04.149724+0000","last_became_active":"2024-04-25T17:08:12.165569+0000","last_became_peered":"2024-04-25T17:08:12.165569+0000","last_unstale":"2024-04-25T17:09:04.149724+0000","last_undegraded":"2024-04-25T17:09:04.149724+0000","last_fullsized":"2024-04-25T17:09:04.149724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:15:16.230457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149511+0000","last_change":"2024-04-25T17:08:11.158927+0000","last_active":"2024-04-25T17:09:04.149511+0000","last_peered":"2024-04-25T17:09:04.149511+0000","last_clean":"2024-04-25T17:09:04.149511+0000","last_became_active":"2024-04-25T17:08:11.158528+0000","last_became_peered":"2024-04-25T17:08:11.158528+0000","last_unstale":"2024-04-25T17:09:04.149511+0000","last_undegraded":"2024-04-25T17:09:04.149511+0000","last_fullsized":"2024-04-25T17:09:04.149511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:50.308627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145716+0000","last_change":"2024-04-25T17:08:12.160776+0000","last_active":"2024-04-25T17:09:04.145716+0000","last_peered":"2024-04-25T17:09:04.145716+0000","last_clean":"2024-04-25T17:09:04.145716+0000","last_became_active":"2024-04-25T17:08:12.160409+0000","last_became_peered":"2024-04-25T17:08:12.160409+0000","last_unstale":"2024-04-25T17:09:04.145716+0000","last_undegraded":"2024-04-25T17:09:04.145716+0000","last_fullsized":"2024-04-25T17:09:04.145716+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:51.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150280+0000","last_change":"2024-04-25T17:08:11.153214+0000","last_active":"2024-04-25T17:09:04.150280+0000","last_peered":"2024-04-25T17:09:04.150280+0000","last_clean":"2024-04-25T17:09:04.150280+0000","last_became_active":"2024-04-25T17:08:11.152728+0000","last_became_peered":"2024-04-25T17:08:11.152728+0000","last_unstale":"2024-04-25T17:09:04.150280+0000","last_undegraded":"2024-04-25T17:09:04.150280+0000","last_fullsized":"2024-04-25T17:09:04.150280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.338549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151034+0000","last_change":"2024-04-25T17:09:04.151034+0000","last_active":"2024-04-25T17:09:04.151034+0000","last_peered":"2024-04-25T17:09:04.151034+0000","last_clean":"2024-04-25T17:09:04.151034+0000","last_became_active":"2024-04-25T17:09:04.150605+0000","last_became_peered":"2024-04-25T17:09:04.150605+0000","last_unstale":"2024-04-25T17:09:04.151034+0000","last_undegraded":"2024-04-25T17:09:04.151034+0000","last_fullsized":"2024-04-25T17:09:04.151034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:32.853957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149135+0000","last_change":"2024-04-25T17:08:11.158995+0000","last_active":"2024-04-25T17:09:04.149135+0000","last_peered":"2024-04-25T17:09:04.149135+0000","last_clean":"2024-04-25T17:09:04.149135+0000","last_became_active":"2024-04-25T17:08:11.158644+0000","last_became_peered":"2024-04-25T17:08:11.158644+0000","last_unstale":"2024-04-25T17:09:04.149135+0000","last_undegraded":"2024-04-25T17:09:04.149135+0000","last_fullsized":"2024-04-25T17:09:04.149135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:22.736433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'5","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.079659+0000","last_change":"2024-04-25T17:08:12.161635+0000","last_active":"2024-04-25T17:09:33.079659+0000","last_peered":"2024-04-25T17:09:33.079659+0000","last_clean":"2024-04-25T17:09:33.079659+0000","last_became_active":"2024-04-25T17:08:12.153728+0000","last_became_peered":"2024-04-25T17:08:12.153728+0000","last_unstale":"2024-04-25T17:09:33.079659+0000","last_undegraded":"2024-04-25T17:09:33.079659+0000","last_fullsized":"2024-04-25T17:09:33.079659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:02.003550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150239+0000","last_change":"2024-04-25T17:08:11.150490+0000","last_active":"2024-04-25T17:09:04.150239+0000","last_peered":"2024-04-25T17:09:04.150239+0000","last_clean":"2024-04-25T17:09:04.150239+0000","last_became_active":"2024-04-25T17:08:11.148391+0000","last_became_peered":"2024-04-25T17:08:11.148391+0000","last_unstale":"2024-04-25T17:09:04.150239+0000","last_undegraded":"2024-04-25T17:09:04.150239+0000","last_fullsized":"2024-04-25T17:09:04.150239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:40.731658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.074958+0000","last_change":"2024-04-25T17:08:12.168136+0000","last_active":"2024-04-25T17:09:33.074958+0000","last_peered":"2024-04-25T17:09:33.074958+0000","last_clean":"2024-04-25T17:09:33.074958+0000","last_became_active":"2024-04-25T17:08:12.167786+0000","last_became_peered":"2024-04-25T17:08:12.167786+0000","last_unstale":"2024-04-25T17:09:33.074958+0000","last_undegraded":"2024-04-25T17:09:33.074958+0000","last_fullsized":"2024-04-25T17:09:33.074958+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:31:22.869503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145916+0000","last_change":"2024-04-25T17:08:12.159952+0000","last_active":"2024-04-25T17:09:04.145916+0000","last_peered":"2024-04-25T17:09:04.145916+0000","last_clean":"2024-04-25T17:09:04.145916+0000","last_became_active":"2024-04-25T17:08:12.159429+0000","last_became_peered":"2024-04-25T17:08:12.159429+0000","last_unstale":"2024-04-25T17:09:04.145916+0000","last_undegraded":"2024-04-25T17:09:04.145916+0000","last_fullsized":"2024-04-25T17:09:04.145916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:13:30.521084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.144707+0000","last_change":"2024-04-25T17:08:11.149085+0000","last_active":"2024-04-25T17:09:04.144707+0000","last_peered":"2024-04-25T17:09:04.144707+0000","last_clean":"2024-04-25T17:09:04.144707+0000","last_became_active":"2024-04-25T17:08:11.148712+0000","last_became_peered":"2024-04-25T17:08:11.148712+0000","last_unstale":"2024-04-25T17:09:04.144707+0000","last_undegraded":"2024-04-25T17:09:04.144707+0000","last_fullsized":"2024-04-25T17:09:04.144707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:07:12.764034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236861+0000","last_change":"2024-04-25T17:08:12.178154+0000","last_active":"2024-04-25T17:09:04.236861+0000","last_peered":"2024-04-25T17:09:04.236861+0000","last_clean":"2024-04-25T17:09:04.236861+0000","last_became_active":"2024-04-25T17:08:12.177773+0000","last_became_peered":"2024-04-25T17:08:12.177773+0000","last_unstale":"2024-04-25T17:09:04.236861+0000","last_undegraded":"2024-04-25T17:09:04.236861+0000","last_fullsized":"2024-04-25T17:09:04.236861+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.962410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145510+0000","last_change":"2024-04-25T17:08:11.152932+0000","last_active":"2024-04-25T17:09:04.145510+0000","last_peered":"2024-04-25T17:09:04.145510+0000","last_clean":"2024-04-25T17:09:04.145510+0000","last_became_active":"2024-04-25T17:08:11.152530+0000","last_became_peered":"2024-04-25T17:08:11.152530+0000","last_unstale":"2024-04-25T17:09:04.145510+0000","last_undegraded":"2024-04-25T17:09:04.145510+0000","last_fullsized":"2024-04-25T17:09:04.145510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:02.654939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149811+0000","last_change":"2024-04-25T17:06:55.951326+0000","last_active":"2024-04-25T17:09:04.149811+0000","last_peered":"2024-04-25T17:09:04.149811+0000","last_clean":"2024-04-25T17:09:04.149811+0000","last_became_active":"2024-04-25T17:06:55.946641+0000","last_became_peered":"2024-04-25T17:06:55.946641+0000","last_unstale":"2024-04-25T17:09:04.149811+0000","last_undegraded":"2024-04-25T17:09:04.149811+0000","last_fullsized":"2024-04-25T17:09:04.149811+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:03:27.394276+0000","last_clean_scrub_stamp":"2024-04-25T17:03:27.394276+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:58.308843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"78'5","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.075220+0000","last_change":"2024-04-25T17:09:04.152927+0000","last_active":"2024-04-25T17:09:33.075220+0000","last_peered":"2024-04-25T17:09:33.075220+0000","last_clean":"2024-04-25T17:09:33.075220+0000","last_became_active":"2024-04-25T17:09:04.152541+0000","last_became_peered":"2024-04-25T17:09:04.152541+0000","last_unstale":"2024-04-25T17:09:33.075220+0000","last_undegraded":"2024-04-25T17:09:33.075220+0000","last_fullsized":"2024-04-25T17:09:33.075220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:30.076215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150651+0000","last_change":"2024-04-25T17:08:11.155941+0000","last_active":"2024-04-25T17:09:04.150651+0000","last_peered":"2024-04-25T17:09:04.150651+0000","last_clean":"2024-04-25T17:09:04.150651+0000","last_became_active":"2024-04-25T17:08:11.155381+0000","last_became_peered":"2024-04-25T17:08:11.155381+0000","last_unstale":"2024-04-25T17:09:04.150651+0000","last_undegraded":"2024-04-25T17:09:04.150651+0000","last_fullsized":"2024-04-25T17:09:04.150651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:28.423462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150036+0000","last_change":"2024-04-25T17:08:11.158960+0000","last_active":"2024-04-25T17:09:04.150036+0000","last_peered":"2024-04-25T17:09:04.150036+0000","last_clean":"2024-04-25T17:09:04.150036+0000","last_became_active":"2024-04-25T17:08:11.158609+0000","last_became_peered":"2024-04-25T17:08:11.158609+0000","last_unstale":"2024-04-25T17:09:04.150036+0000","last_undegraded":"2024-04-25T17:09:04.150036+0000","last_fullsized":"2024-04-25T17:09:04.150036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:19.087415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"78'9","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.075404+0000","last_change":"2024-04-25T17:08:12.161483+0000","last_active":"2024-04-25T17:09:33.075404+0000","last_peered":"2024-04-25T17:09:33.075404+0000","last_clean":"2024-04-25T17:09:33.075404+0000","last_became_active":"2024-04-25T17:08:12.158586+0000","last_became_peered":"2024-04-25T17:08:12.158586+0000","last_unstale":"2024-04-25T17:09:33.075404+0000","last_undegraded":"2024-04-25T17:09:33.075404+0000","last_fullsized":"2024-04-25T17:09:33.075404+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.551328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'6","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149766+0000","last_change":"2024-04-25T17:08:11.161497+0000","last_active":"2024-04-25T17:09:04.149766+0000","last_peered":"2024-04-25T17:09:04.149766+0000","last_clean":"2024-04-25T17:09:04.149766+0000","last_became_active":"2024-04-25T17:08:11.160990+0000","last_became_peered":"2024-04-25T17:08:11.160990+0000","last_unstale":"2024-04-25T17:09:04.149766+0000","last_undegraded":"2024-04-25T17:09:04.149766+0000","last_fullsized":"2024-04-25T17:09:04.149766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:59.682133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236920+0000","last_change":"2024-04-25T17:08:12.171214+0000","last_active":"2024-04-25T17:09:04.236920+0000","last_peered":"2024-04-25T17:09:04.236920+0000","last_clean":"2024-04-25T17:09:04.236920+0000","last_became_active":"2024-04-25T17:08:12.170877+0000","last_became_peered":"2024-04-25T17:08:12.170877+0000","last_unstale":"2024-04-25T17:09:04.236920+0000","last_undegraded":"2024-04-25T17:09:04.236920+0000","last_fullsized":"2024-04-25T17:09:04.236920+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:18.442691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145995+0000","last_change":"2024-04-25T17:08:11.154819+0000","last_active":"2024-04-25T17:09:04.145995+0000","last_peered":"2024-04-25T17:09:04.145995+0000","last_clean":"2024-04-25T17:09:04.145995+0000","last_became_active":"2024-04-25T17:08:11.154006+0000","last_became_peered":"2024-04-25T17:08:11.154006+0000","last_unstale":"2024-04-25T17:09:04.145995+0000","last_undegraded":"2024-04-25T17:09:04.145995+0000","last_fullsized":"2024-04-25T17:09:04.145995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.063058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.076372+0000","last_change":"2024-04-25T17:08:12.167301+0000","last_active":"2024-04-25T17:09:33.076372+0000","last_peered":"2024-04-25T17:09:33.076372+0000","last_clean":"2024-04-25T17:09:33.076372+0000","last_became_active":"2024-04-25T17:08:12.166944+0000","last_became_peered":"2024-04-25T17:08:12.166944+0000","last_unstale":"2024-04-25T17:09:33.076372+0000","last_undegraded":"2024-04-25T17:09:33.076372+0000","last_fullsized":"2024-04-25T17:09:33.076372+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:46.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236606+0000","last_change":"2024-04-25T17:08:11.156616+0000","last_active":"2024-04-25T17:09:04.236606+0000","last_peered":"2024-04-25T17:09:04.236606+0000","last_clean":"2024-04-25T17:09:04.236606+0000","last_became_active":"2024-04-25T17:08:11.156256+0000","last_became_peered":"2024-04-25T17:08:11.156256+0000","last_unstale":"2024-04-25T17:09:04.236606+0000","last_undegraded":"2024-04-25T17:09:04.236606+0000","last_fullsized":"2024-04-25T17:09:04.236606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:34:37.539773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149432+0000","last_change":"2024-04-25T17:08:12.175216+0000","last_active":"2024-04-25T17:09:04.149432+0000","last_peered":"2024-04-25T17:09:04.149432+0000","last_clean":"2024-04-25T17:09:04.149432+0000","last_became_active":"2024-04-25T17:08:12.174521+0000","last_became_peered":"2024-04-25T17:08:12.174521+0000","last_unstale":"2024-04-25T17:09:04.149432+0000","last_undegraded":"2024-04-25T17:09:04.149432+0000","last_fullsized":"2024-04-25T17:09:04.149432+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.322733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236262+0000","last_change":"2024-04-25T17:08:11.162287+0000","last_active":"2024-04-25T17:09:04.236262+0000","last_peered":"2024-04-25T17:09:04.236262+0000","last_clean":"2024-04-25T17:09:04.236262+0000","last_became_active":"2024-04-25T17:08:11.158297+0000","last_became_peered":"2024-04-25T17:08:11.158297+0000","last_unstale":"2024-04-25T17:09:04.236262+0000","last_undegraded":"2024-04-25T17:09:04.236262+0000","last_fullsized":"2024-04-25T17:09:04.236262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:15.758880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147484+0000","last_change":"2024-04-25T17:08:12.163535+0000","last_active":"2024-04-25T17:09:04.147484+0000","last_peered":"2024-04-25T17:09:04.147484+0000","last_clean":"2024-04-25T17:09:04.147484+0000","last_became_active":"2024-04-25T17:08:12.162848+0000","last_became_peered":"2024-04-25T17:08:12.162848+0000","last_unstale":"2024-04-25T17:09:04.147484+0000","last_undegraded":"2024-04-25T17:09:04.147484+0000","last_fullsized":"2024-04-25T17:09:04.147484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:50.288869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.238585+0000","last_change":"2024-04-25T17:09:04.238038+0000","last_active":"2024-04-25T17:09:04.238585+0000","last_peered":"2024-04-25T17:09:04.238585+0000","last_clean":"2024-04-25T17:09:04.238585+0000","last_became_active":"2024-04-25T17:09:04.237635+0000","last_became_peered":"2024-04-25T17:09:04.237635+0000","last_unstale":"2024-04-25T17:09:04.238585+0000","last_undegraded":"2024-04-25T17:09:04.238585+0000","last_fullsized":"2024-04-25T17:09:04.238585+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:14.025433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236530+0000","last_change":"2024-04-25T17:08:12.170828+0000","last_active":"2024-04-25T17:09:04.236530+0000","last_peered":"2024-04-25T17:09:04.236530+0000","last_clean":"2024-04-25T17:09:04.236530+0000","last_became_active":"2024-04-25T17:08:12.170479+0000","last_became_peered":"2024-04-25T17:08:12.170479+0000","last_unstale":"2024-04-25T17:09:04.236530+0000","last_undegraded":"2024-04-25T17:09:04.236530+0000","last_fullsized":"2024-04-25T17:09:04.236530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.947118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":32,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.157988+0000","last_change":"2024-04-25T17:09:04.157988+0000","last_active":"2024-04-25T17:09:04.157988+0000","last_peered":"2024-04-25T17:09:04.157988+0000","last_clean":"2024-04-25T17:09:04.157988+0000","last_became_active":"2024-04-25T17:09:04.153139+0000","last_became_peered":"2024-04-25T17:09:04.153139+0000","last_unstale":"2024-04-25T17:09:04.157988+0000","last_undegraded":"2024-04-25T17:09:04.157988+0000","last_fullsized":"2024-04-25T17:09:04.157988+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:57.969869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150547+0000","last_change":"2024-04-25T17:08:12.162032+0000","last_active":"2024-04-25T17:09:04.150547+0000","last_peered":"2024-04-25T17:09:04.150547+0000","last_clean":"2024-04-25T17:09:04.150547+0000","last_became_active":"2024-04-25T17:08:12.161017+0000","last_became_peered":"2024-04-25T17:08:12.161017+0000","last_unstale":"2024-04-25T17:09:04.150547+0000","last_undegraded":"2024-04-25T17:09:04.150547+0000","last_fullsized":"2024-04-25T17:09:04.150547+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.472984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151417+0000","last_change":"2024-04-25T17:08:11.149550+0000","last_active":"2024-04-25T17:09:04.151417+0000","last_peered":"2024-04-25T17:09:04.151417+0000","last_clean":"2024-04-25T17:09:04.151417+0000","last_became_active":"2024-04-25T17:08:11.149082+0000","last_became_peered":"2024-04-25T17:08:11.149082+0000","last_unstale":"2024-04-25T17:09:04.151417+0000","last_undegraded":"2024-04-25T17:09:04.151417+0000","last_fullsized":"2024-04-25T17:09:04.151417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:19:14.565877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150444+0000","last_change":"2024-04-25T17:08:12.160331+0000","last_active":"2024-04-25T17:09:04.150444+0000","last_peered":"2024-04-25T17:09:04.150444+0000","last_clean":"2024-04-25T17:09:04.150444+0000","last_became_active":"2024-04-25T17:08:12.159931+0000","last_became_peered":"2024-04-25T17:08:12.159931+0000","last_unstale":"2024-04-25T17:09:04.150444+0000","last_undegraded":"2024-04-25T17:09:04.150444+0000","last_fullsized":"2024-04-25T17:09:04.150444+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:37.246974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.146028+0000","last_change":"2024-04-25T17:08:11.163118+0000","last_active":"2024-04-25T17:09:04.146028+0000","last_peered":"2024-04-25T17:09:04.146028+0000","last_clean":"2024-04-25T17:09:04.146028+0000","last_became_active":"2024-04-25T17:08:11.161691+0000","last_became_peered":"2024-04-25T17:08:11.161691+0000","last_unstale":"2024-04-25T17:09:04.146028+0000","last_undegraded":"2024-04-25T17:09:04.146028+0000","last_fullsized":"2024-04-25T17:09:04.146028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:32.006098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'1","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.149868+0000","last_change":"2024-04-25T17:08:12.168387+0000","last_active":"2024-04-25T17:09:04.149868+0000","last_peered":"2024-04-25T17:09:04.149868+0000","last_clean":"2024-04-25T17:09:04.149868+0000","last_became_active":"2024-04-25T17:08:12.168008+0000","last_became_peered":"2024-04-25T17:08:12.168008+0000","last_unstale":"2024-04-25T17:09:04.149868+0000","last_undegraded":"2024-04-25T17:09:04.149868+0000","last_fullsized":"2024-04-25T17:09:04.149868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:08.438945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145175+0000","last_change":"2024-04-25T17:08:11.149181+0000","last_active":"2024-04-25T17:09:04.145175+0000","last_peered":"2024-04-25T17:09:04.145175+0000","last_clean":"2024-04-25T17:09:04.145175+0000","last_became_active":"2024-04-25T17:08:11.148847+0000","last_became_peered":"2024-04-25T17:08:11.148847+0000","last_unstale":"2024-04-25T17:09:04.145175+0000","last_undegraded":"2024-04-25T17:09:04.145175+0000","last_fullsized":"2024-04-25T17:09:04.145175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:10.119218+0000","last_clean_scrub_stamp":"2024-04-25T17:08:10.119218+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:40.720578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.150028+0000","last_change":"2024-04-25T17:08:12.162173+0000","last_active":"2024-04-25T17:09:04.150028+0000","last_peered":"2024-04-25T17:09:04.150028+0000","last_clean":"2024-04-25T17:09:04.150028+0000","last_became_active":"2024-04-25T17:08:12.161818+0000","last_became_peered":"2024-04-25T17:08:12.161818+0000","last_unstale":"2024-04-25T17:09:04.150028+0000","last_undegraded":"2024-04-25T17:09:04.150028+0000","last_fullsized":"2024-04-25T17:09:04.150028+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:28.872937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147567+0000","last_change":"2024-04-25T17:08:12.165542+0000","last_active":"2024-04-25T17:09:04.147567+0000","last_peered":"2024-04-25T17:09:04.147567+0000","last_clean":"2024-04-25T17:09:04.147567+0000","last_became_active":"2024-04-25T17:08:12.165190+0000","last_became_peered":"2024-04-25T17:08:12.165190+0000","last_unstale":"2024-04-25T17:09:04.147567+0000","last_undegraded":"2024-04-25T17:09:04.147567+0000","last_fullsized":"2024-04-25T17:09:04.147567+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:51.650953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151132+0000","last_change":"2024-04-25T17:08:12.161429+0000","last_active":"2024-04-25T17:09:04.151132+0000","last_peered":"2024-04-25T17:09:04.151132+0000","last_clean":"2024-04-25T17:09:04.151132+0000","last_became_active":"2024-04-25T17:08:12.159184+0000","last_became_peered":"2024-04-25T17:08:12.159184+0000","last_unstale":"2024-04-25T17:09:04.151132+0000","last_undegraded":"2024-04-25T17:09:04.151132+0000","last_fullsized":"2024-04-25T17:09:04.151132+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:56:21.366775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.151032+0000","last_change":"2024-04-25T17:08:12.169948+0000","last_active":"2024-04-25T17:09:04.151032+0000","last_peered":"2024-04-25T17:09:04.151032+0000","last_clean":"2024-04-25T17:09:04.151032+0000","last_became_active":"2024-04-25T17:08:12.169592+0000","last_became_peered":"2024-04-25T17:08:12.169592+0000","last_unstale":"2024-04-25T17:09:04.151032+0000","last_undegraded":"2024-04-25T17:09:04.151032+0000","last_fullsized":"2024-04-25T17:09:04.151032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:22.749954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"78'7","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.077924+0000","last_change":"2024-04-25T17:08:12.169785+0000","last_active":"2024-04-25T17:09:33.077924+0000","last_peered":"2024-04-25T17:09:33.077924+0000","last_clean":"2024-04-25T17:09:33.077924+0000","last_became_active":"2024-04-25T17:08:12.169226+0000","last_became_peered":"2024-04-25T17:08:12.169226+0000","last_unstale":"2024-04-25T17:09:33.077924+0000","last_undegraded":"2024-04-25T17:09:33.077924+0000","last_fullsized":"2024-04-25T17:09:33.077924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.232112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"78'4","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.079482+0000","last_change":"2024-04-25T17:08:12.157992+0000","last_active":"2024-04-25T17:09:33.079482+0000","last_peered":"2024-04-25T17:09:33.079482+0000","last_clean":"2024-04-25T17:09:33.079482+0000","last_became_active":"2024-04-25T17:08:12.157636+0000","last_became_peered":"2024-04-25T17:08:12.157636+0000","last_unstale":"2024-04-25T17:09:33.079482+0000","last_undegraded":"2024-04-25T17:09:33.079482+0000","last_fullsized":"2024-04-25T17:09:33.079482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:50.530074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147926+0000","last_change":"2024-04-25T17:08:12.160977+0000","last_active":"2024-04-25T17:09:04.147926+0000","last_peered":"2024-04-25T17:09:04.147926+0000","last_clean":"2024-04-25T17:09:04.147926+0000","last_became_active":"2024-04-25T17:08:12.160571+0000","last_became_peered":"2024-04-25T17:08:12.160571+0000","last_unstale":"2024-04-25T17:09:04.147926+0000","last_undegraded":"2024-04-25T17:09:04.147926+0000","last_fullsized":"2024-04-25T17:09:04.147926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:32.882628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.153530+0000","last_change":"2024-04-25T17:08:12.158566+0000","last_active":"2024-04-25T17:09:04.153530+0000","last_peered":"2024-04-25T17:09:04.153530+0000","last_clean":"2024-04-25T17:09:04.153530+0000","last_became_active":"2024-04-25T17:08:12.157911+0000","last_became_peered":"2024-04-25T17:08:12.157911+0000","last_unstale":"2024-04-25T17:09:04.153530+0000","last_undegraded":"2024-04-25T17:09:04.153530+0000","last_fullsized":"2024-04-25T17:09:04.153530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:49.665779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236889+0000","last_change":"2024-04-25T17:08:12.174931+0000","last_active":"2024-04-25T17:09:04.236889+0000","last_peered":"2024-04-25T17:09:04.236889+0000","last_clean":"2024-04-25T17:09:04.236889+0000","last_became_active":"2024-04-25T17:08:12.174571+0000","last_became_peered":"2024-04-25T17:08:12.174571+0000","last_unstale":"2024-04-25T17:09:04.236889+0000","last_undegraded":"2024-04-25T17:09:04.236889+0000","last_fullsized":"2024-04-25T17:09:04.236889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:55.832578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.145585+0000","last_change":"2024-04-25T17:08:12.165592+0000","last_active":"2024-04-25T17:09:04.145585+0000","last_peered":"2024-04-25T17:09:04.145585+0000","last_clean":"2024-04-25T17:09:04.145585+0000","last_became_active":"2024-04-25T17:08:12.165273+0000","last_became_peered":"2024-04-25T17:08:12.165273+0000","last_unstale":"2024-04-25T17:09:04.145585+0000","last_undegraded":"2024-04-25T17:09:04.145585+0000","last_fullsized":"2024-04-25T17:09:04.145585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:25.199657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.236235+0000","last_change":"2024-04-25T17:08:12.174095+0000","last_active":"2024-04-25T17:09:04.236235+0000","last_peered":"2024-04-25T17:09:04.236235+0000","last_clean":"2024-04-25T17:09:04.236235+0000","last_became_active":"2024-04-25T17:08:12.172786+0000","last_became_peered":"2024-04-25T17:08:12.172786+0000","last_unstale":"2024-04-25T17:09:04.236235+0000","last_undegraded":"2024-04-25T17:09:04.236235+0000","last_fullsized":"2024-04-25T17:09:04.236235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.736300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"78'10","reported_seq":27,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.091355+0000","last_change":"2024-04-25T17:08:12.158772+0000","last_active":"2024-04-25T17:09:33.091355+0000","last_peered":"2024-04-25T17:09:33.091355+0000","last_clean":"2024-04-25T17:09:33.091355+0000","last_became_active":"2024-04-25T17:08:12.158417+0000","last_became_peered":"2024-04-25T17:08:12.158417+0000","last_unstale":"2024-04-25T17:09:33.091355+0000","last_undegraded":"2024-04-25T17:09:33.091355+0000","last_fullsized":"2024-04-25T17:09:33.091355+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:21.837887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"78'11","reported_seq":28,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:33.078966+0000","last_change":"2024-04-25T17:08:12.162384+0000","last_active":"2024-04-25T17:09:33.078966+0000","last_peered":"2024-04-25T17:09:33.078966+0000","last_clean":"2024-04-25T17:09:33.078966+0000","last_became_active":"2024-04-25T17:08:12.161997+0000","last_became_peered":"2024-04-25T17:08:12.161997+0000","last_unstale":"2024-04-25T17:09:33.078966+0000","last_undegraded":"2024-04-25T17:09:33.078966+0000","last_fullsized":"2024-04-25T17:09:33.078966+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:17.358171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-04-25T17:09:04.147625+0000","last_change":"2024-04-25T17:08:12.159015+0000","last_active":"2024-04-25T17:09:04.147625+0000","last_peered":"2024-04-25T17:09:04.147625+0000","last_clean":"2024-04-25T17:09:04.147625+0000","last_became_active":"2024-04-25T17:08:12.157999+0000","last_became_peered":"2024-04-25T17:08:12.157999+0000","last_unstale":"2024-04-25T17:09:04.147625+0000","last_undegraded":"2024-04-25T17:09:04.147625+0000","last_fullsized":"2024-04-25T17:09:04.147625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T17:08:11.128828+0000","last_clean_scrub_stamp":"2024-04-25T17:08:11.128828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:00.291221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280965,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62093,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":172,"num_read_kb":148,"num_write":78,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":186030,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":67,"ondisk_log_size":67,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678045,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":835584,"data_stored":406301,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874275,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1058275,"data_compressed":43374,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037801,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":406883,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":406323,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430259,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":997368,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":406418,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":467929,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":406391,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":467381,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":406639,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986243,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":406323,"data_compressed":5801,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":996968,"data_compressed":41531,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34105,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35730,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61877,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61080,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61489,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T17:09:35.745 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T17:09:35.745 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T17:09:35.745 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T17:09:35.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T17:09:35.982 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:36.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:36 smithi096 ceph-mon[29131]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:36.360 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:36 smithi096 ceph-mon[29131]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:09:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:36 smithi097 ceph-mon[32602]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:36.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:36 smithi097 ceph-mon[32602]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:09:36.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:36 smithi171 ceph-mon[33189]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:36.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:36 smithi171 ceph-mon[33189]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T17:09:37.404 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:37 smithi096 ceph-mon[29131]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:09:37.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:37 smithi097 ceph-mon[32602]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:09:37.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:37 smithi171 ceph-mon[33189]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 -- 172.21.15.96:0/483898412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 msgr2=0x7f6574111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 --2- 172.21.15.96:0/483898412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f6574111090 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f656c00b0d0 tx=0x7f656c00b3e0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 -- 172.21.15.96:0/483898412 shutdown_connections 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 --2- 172.21.15.96:0/483898412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f6574111090 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 --2- 172.21.15.96:0/483898412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65741044b0 0x7f65741048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 --2- 172.21.15.96:0/483898412 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6574108de0 0x7f65741091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 -- 172.21.15.96:0/483898412 >> 172.21.15.96:0/483898412 conn(0x7f65740ffda0 msgr2=0x7f65741021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 -- 172.21.15.96:0/483898412 shutdown_connections 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.646+0000 7f657a33b700 1 -- 172.21.15.96:0/483898412 wait complete. 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 Processor -- start 2024-04-25T17:09:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 -- start start 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65741044b0 0x7f657419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6574108de0 0x7f657419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f657419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6574114450 con 0x7f6574109730 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f65741142d0 con 0x7f6574108de0 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.647+0000 7f657a33b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65741145d0 con 0x7f65741044b0 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f657419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f657419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36394/0 (socket says 172.21.15.96:36394) 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 -- 172.21.15.96:0/2094425770 learned_addr learned my addr 172.21.15.96:0/2094425770 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65737fe700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6574108de0 0x7f657419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f6573fff700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65741044b0 0x7f657419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 -- 172.21.15.96:0/2094425770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65741044b0 msgr2=0x7f657419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65741044b0 0x7f657419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 -- 172.21.15.96:0/2094425770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6574108de0 msgr2=0x7f657419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6574108de0 0x7f657419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 -- 172.21.15.96:0/2094425770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657419fc90 con 0x7f6574109730 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65788d8700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f657419f5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65717fa700 1 -- 172.21.15.96:0/2094425770 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c01f070 con 0x7f6574109730 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657419ff20 con 0x7f6574109730 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65717fa700 1 -- 172.21.15.96:0/2094425770 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c003c00 con 0x7f6574109730 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f65717fa700 1 -- 172.21.15.96:0/2094425770 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c01b7b0 con 0x7f6574109730 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.648+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741a78a0 con 0x7f6574109730 2024-04-25T17:09:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.649+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6574105960 con 0x7f6574109730 2024-04-25T17:09:37.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.651+0000 7f65717fa700 1 -- 172.21.15.96:0/2094425770 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f656c022070 con 0x7f6574109730 2024-04-25T17:09:37.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.651+0000 7f65717fa700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f655c0afb80 0x7f655c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:37.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.651+0000 7f6573fff700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f655c0afb80 0x7f655c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:37.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.652+0000 7f65717fa700 1 -- 172.21.15.96:0/2094425770 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f656c00f020 con 0x7f6574109730 2024-04-25T17:09:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.652+0000 7f6573fff700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f655c0afb80 0x7f655c0b2090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:37.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.655+0000 7f65717fa700 1 -- 172.21.15.96:0/2094425770 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f656c029020 con 0x7f6574109730 2024-04-25T17:09:37.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.898+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6574051190 con 0x7f6574109730 2024-04-25T17:09:37.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.899+0000 7f65717fa700 1 -- 172.21.15.96:0/2094425770 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f656c0a8f70 con 0x7f6574109730 2024-04-25T17:09:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:37.899 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f655c0afb80 msgr2=0x7f655c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f655c0afb80 0x7f655c0b2090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564008040 comp rx=0 tx=0).stop 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 msgr2=0x7f657419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f657419f5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c007a20 comp rx=0 tx=0).stop 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 shutdown_connections 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f655c0afb80 0x7f655c0b2090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6574109730 0x7f657419f5f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65741044b0 0x7f657419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 --2- 172.21.15.96:0/2094425770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6574108de0 0x7f657419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 >> 172.21.15.96:0/2094425770 conn(0x7f65740ffda0 msgr2=0x7f6574108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:37.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 shutdown_connections 2024-04-25T17:09:37.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:37.901+0000 7f657a33b700 1 -- 172.21.15.96:0/2094425770 wait complete. 2024-04-25T17:09:38.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:38 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2094425770' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T17:09:38.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:38 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2094425770' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T17:09:38.524 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T17:09:38.524 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:09:38.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:38 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2094425770' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T17:09:38.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 -- 172.21.15.96:0/2212182996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102a00 msgr2=0x7fc080103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 --2- 172.21.15.96:0/2212182996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102a00 0x7fc080103700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009ca0 tx=0x7fc07c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 -- 172.21.15.96:0/2212182996 shutdown_connections 2024-04-25T17:09:38.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 --2- 172.21.15.96:0/2212182996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc080108940 0x7fc08010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 --2- 172.21.15.96:0/2212182996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc080103c40 0x7fc080104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 --2- 172.21.15.96:0/2212182996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080102a00 0x7fc080103700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 -- 172.21.15.96:0/2212182996 >> 172.21.15.96:0/2212182996 conn(0x7fc0800fdbf0 msgr2=0x7fc0801000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 -- 172.21.15.96:0/2212182996 shutdown_connections 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.634+0000 7fc088442640 1 -- 172.21.15.96:0/2212182996 wait complete. 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 Processor -- start 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 -- start start 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc080102a00 0x7fc08019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc080103c40 0x7fc08019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080108940 0x7fc0801a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010f630 con 0x7fc080108940 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010f4b0 con 0x7fc080102a00 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc088442640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc08010f7b0 con 0x7fc080103c40 2024-04-25T17:09:38.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080108940 0x7fc0801a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080108940 0x7fc0801a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36406/0 (socket says 172.21.15.96:36406) 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 -- 172.21.15.96:0/1502648283 learned_addr learned my addr 172.21.15.96:0/1502648283 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0861de640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc080102a00 0x7fc08019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0859dd640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc080103c40 0x7fc08019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 -- 172.21.15.96:0/1502648283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc080103c40 msgr2=0x7fc08019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc080103c40 0x7fc08019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 -- 172.21.15.96:0/1502648283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc080102a00 msgr2=0x7fc08019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc080102a00 0x7fc08019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0869df640 1 -- 172.21.15.96:0/1502648283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0801a1cd0 con 0x7fc080108940 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.635+0000 7fc0859dd640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc080103c40 0x7fc08019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.636+0000 7fc0869df640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080108940 0x7fc0801a1710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c970 tx=0x7fc07800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.636+0000 7fc0777fe640 1 -- 172.21.15.96:0/1502648283 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078007bb0 con 0x7fc080108940 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.636+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0801a1fc0 con 0x7fc080108940 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.636+0000 7fc0777fe640 1 -- 172.21.15.96:0/1502648283 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc078007d50 con 0x7fc080108940 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.636+0000 7fc0777fe640 1 -- 172.21.15.96:0/1502648283 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078018770 con 0x7fc080108940 2024-04-25T17:09:38.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.636+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0801a2420 con 0x7fc080108940 2024-04-25T17:09:38.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.637+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc080107750 con 0x7fc080108940 2024-04-25T17:09:38.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.638+0000 7fc0777fe640 1 -- 172.21.15.96:0/1502648283 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc07800f990 con 0x7fc080108940 2024-04-25T17:09:38.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.639+0000 7fc0777fe640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc06c0afaf0 0x7fc06c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.639+0000 7fc0861de640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc06c0afaf0 0x7fc06c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.639+0000 7fc0777fe640 1 -- 172.21.15.96:0/1502648283 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fc0780ded30 con 0x7fc080108940 2024-04-25T17:09:38.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.639+0000 7fc0861de640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc06c0afaf0 0x7fc06c0b2000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000f80 tx=0x7fc07c009420 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:38.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.642+0000 7fc0777fe640 1 -- 172.21.15.96:0/1502648283 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc078014030 con 0x7fc080108940 2024-04-25T17:09:38.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.821+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc08002dd20 con 0x7fc080108940 2024-04-25T17:09:38.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.823+0000 7fc0777fe640 1 -- 172.21.15.96:0/1502648283 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22594 (secure 0 0 0) 0x7fc0780a79a0 con 0x7fc080108940 2024-04-25T17:09:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:38.823 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:09:04.141423+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc06c0afaf0 msgr2=0x7fc06c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc06c0afaf0 0x7fc06c0b2000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc07c000f80 tx=0x7fc07c009420 comp rx=0 tx=0).stop 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080108940 msgr2=0x7fc0801a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080108940 0x7fc0801a1710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc07800c970 tx=0x7fc07800ce30 comp rx=0 tx=0).stop 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 shutdown_connections 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc080108940 0x7fc0801a1710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc06c0afaf0 0x7fc06c0b2000 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc080103c40 0x7fc08019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 --2- 172.21.15.96:0/1502648283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc080102a00 0x7fc08019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 >> 172.21.15.96:0/1502648283 conn(0x7fc0800fdbf0 msgr2=0x7fc080104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 shutdown_connections 2024-04-25T17:09:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.825+0000 7fc088442640 1 -- 172.21.15.96:0/1502648283 wait complete. 2024-04-25T17:09:38.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T17:08:09.266201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T17:08:10.452964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:09:38.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 -- 172.21.15.96:0/4238958943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24104480 msgr2=0x7f6a2410abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 --2- 172.21.15.96:0/4238958943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24104480 0x7f6a2410abc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009900 tx=0x7f6a1402a410 comp rx=0 tx=0).stop 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 -- 172.21.15.96:0/4238958943 shutdown_connections 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 --2- 172.21.15.96:0/4238958943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24104480 0x7f6a2410abc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 --2- 172.21.15.96:0/4238958943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a24103940 0x7f6a24103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 --2- 172.21.15.96:0/4238958943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a24102f60 0x7f6a24103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 -- 172.21.15.96:0/4238958943 >> 172.21.15.96:0/4238958943 conn(0x7f6a240fd380 msgr2=0x7f6a240ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 -- 172.21.15.96:0/4238958943 shutdown_connections 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.947+0000 7f6a28a42640 1 -- 172.21.15.96:0/4238958943 wait complete. 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 Processor -- start 2024-04-25T17:09:38.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 -- start start 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a24102f60 0x7f6a24199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24103940 0x7f6a24199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a24104480 0x7f6a241a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e530 con 0x7f6a24103940 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e3b0 con 0x7f6a24102f60 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a28a42640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e6b0 con 0x7f6a24104480 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24103940 0x7f6a24199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24103940 0x7f6a24199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36436/0 (socket says 172.21.15.96:36436) 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 -- 172.21.15.96:0/1781914558 learned_addr learned my addr 172.21.15.96:0/1781914558 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a2259c640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a24102f60 0x7f6a24199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a22d9d640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a24104480 0x7f6a241a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 -- 172.21.15.96:0/1781914558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a24104480 msgr2=0x7f6a241a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a24104480 0x7f6a241a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 -- 172.21.15.96:0/1781914558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a24102f60 msgr2=0x7f6a24199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a24102f60 0x7f6a24199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.948+0000 7f6a21d9b640 1 -- 172.21.15.96:0/1781914558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a241a0ad0 con 0x7f6a24103940 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.949+0000 7f6a21d9b640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24103940 0x7f6a24199580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c920 tx=0x7f6a1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.949+0000 7f6a137fe640 1 -- 172.21.15.96:0/1781914558 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18007c70 con 0x7f6a24103940 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.949+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241a0dc0 con 0x7f6a24103940 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.949+0000 7f6a137fe640 1 -- 172.21.15.96:0/1781914558 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a18007e10 con 0x7f6a24103940 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.949+0000 7f6a137fe640 1 -- 172.21.15.96:0/1781914558 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18018820 con 0x7f6a24103940 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.949+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a1230 con 0x7f6a24103940 2024-04-25T17:09:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.950+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a24109910 con 0x7f6a24103940 2024-04-25T17:09:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.952+0000 7f6a137fe640 1 -- 172.21.15.96:0/1781914558 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6a1800f450 con 0x7f6a24103940 2024-04-25T17:09:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.953+0000 7f6a137fe640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6a0c0afaf0 0x7f6a0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.953+0000 7f6a2259c640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6a0c0afaf0 0x7f6a0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.953+0000 7f6a137fe640 1 -- 172.21.15.96:0/1781914558 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f6a180df000 con 0x7f6a24103940 2024-04-25T17:09:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.953+0000 7f6a2259c640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6a0c0afaf0 0x7f6a0c0b2000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a2406b4e0 tx=0x7f6a1c0075b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:38.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:38.956+0000 7f6a137fe640 1 -- 172.21.15.96:0/1781914558 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a18014030 con 0x7f6a24103940 2024-04-25T17:09:39.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.131+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a2402dce0 con 0x7f6a24103940 2024-04-25T17:09:39.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.132+0000 7f6a137fe640 1 -- 172.21.15.96:0/1781914558 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f6a180a7c70 con 0x7f6a24103940 2024-04-25T17:09:39.132 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:09:39.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6a0c0afaf0 msgr2=0x7f6a0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6a0c0afaf0 0x7f6a0c0b2000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a2406b4e0 tx=0x7f6a1c0075b0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24103940 msgr2=0x7f6a24199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24103940 0x7f6a24199580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c920 tx=0x7f6a1800cde0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 shutdown_connections 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6a0c0afaf0 0x7f6a0c0b2000 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a24104480 0x7f6a241a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a24103940 0x7f6a24199580 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 --2- 172.21.15.96:0/1781914558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a24102f60 0x7f6a24199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 >> 172.21.15.96:0/1781914558 conn(0x7f6a240fd380 msgr2=0x7f6a24106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 shutdown_connections 2024-04-25T17:09:39.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.134+0000 7f6a28a42640 1 -- 172.21.15.96:0/1781914558 wait complete. 2024-04-25T17:09:39.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T17:09:39.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 -- 172.21.15.96:0/4139467849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8084102260 msgr2=0x7f808410bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 --2- 172.21.15.96:0/4139467849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8084102260 0x7f808410bcc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f807c009ca0 tx=0x7f807c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 -- 172.21.15.96:0/4139467849 shutdown_connections 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 --2- 172.21.15.96:0/4139467849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8084102260 0x7f808410bcc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 --2- 172.21.15.96:0/4139467849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8084101760 0x7f8084101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 --2- 172.21.15.96:0/4139467849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80841050b0 0x7f80841054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 -- 172.21.15.96:0/4139467849 >> 172.21.15.96:0/4139467849 conn(0x7f80840fd380 msgr2=0x7f80840ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 -- 172.21.15.96:0/4139467849 shutdown_connections 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.257+0000 7f808a3b2640 1 -- 172.21.15.96:0/4139467849 wait complete. 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 Processor -- start 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 -- start start 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084101760 0x7f808419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8084102260 0x7f808419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80841050b0 0x7f80841a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f808410f640 con 0x7f80841050b0 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f808410f4c0 con 0x7f8084101760 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808a3b2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f808410f7c0 con 0x7f8084102260 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808894f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80841050b0 0x7f80841a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808894f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80841050b0 0x7f80841a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36458/0 (socket says 172.21.15.96:36458) 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.258+0000 7f808894f640 1 -- 172.21.15.96:0/3061337414 learned_addr learned my addr 172.21.15.96:0/3061337414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f8083fff640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084101760 0x7f808419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f80837fe640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8084102260 0x7f808419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808894f640 1 -- 172.21.15.96:0/3061337414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8084102260 msgr2=0x7f808419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808894f640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8084102260 0x7f808419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808894f640 1 -- 172.21.15.96:0/3061337414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084101760 msgr2=0x7f808419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808894f640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084101760 0x7f808419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808894f640 1 -- 172.21.15.96:0/3061337414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80841a1d30 con 0x7f80841050b0 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808894f640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80841050b0 0x7f80841a16c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f807c0097b0 tx=0x7f807c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f80817fa640 1 -- 172.21.15.96:0/3061337414 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c004240 con 0x7f80841050b0 2024-04-25T17:09:39.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f80817fa640 1 -- 172.21.15.96:0/3061337414 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807c02f380 con 0x7f80841050b0 2024-04-25T17:09:39.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80841a1fc0 con 0x7f80841050b0 2024-04-25T17:09:39.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f80817fa640 1 -- 172.21.15.96:0/3061337414 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c02fc30 con 0x7f80841050b0 2024-04-25T17:09:39.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.259+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a2420 con 0x7f80841050b0 2024-04-25T17:09:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.260+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808406a190 con 0x7f80841050b0 2024-04-25T17:09:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.261+0000 7f80817fa640 1 -- 172.21.15.96:0/3061337414 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f807c0043e0 con 0x7f80841050b0 2024-04-25T17:09:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.262+0000 7f80817fa640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f806c0afaf0 0x7f806c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.262+0000 7f8083fff640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f806c0afaf0 0x7f806c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.263+0000 7f80817fa640 1 -- 172.21.15.96:0/3061337414 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f807c100510 con 0x7f80841050b0 2024-04-25T17:09:39.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.263+0000 7f8083fff640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f806c0afaf0 0x7f806c0b2000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f808406b2d0 tx=0x7f807800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:39.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.266+0000 7f80817fa640 1 -- 172.21.15.96:0/3061337414 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f808406a190 con 0x7f80841050b0 2024-04-25T17:09:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:39 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1502648283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:39 smithi096 ceph-mon[29131]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:39 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1781914558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:09:39.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.443+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f808402dbd0 con 0x7f80841050b0 2024-04-25T17:09:39.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.443+0000 7f80817fa640 1 -- 172.21.15.96:0/3061337414 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f807c0c90b0 con 0x7f80841050b0 2024-04-25T17:09:39.443 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T17:09:39.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.445+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f806c0afaf0 msgr2=0x7f806c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.445+0000 7f808a3b2640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f806c0afaf0 0x7f806c0b2000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f808406b2d0 tx=0x7f807800b390 comp rx=0 tx=0).stop 2024-04-25T17:09:39.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.445+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80841050b0 msgr2=0x7f80841a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.445+0000 7f808a3b2640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80841050b0 0x7f80841a16c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f807c0097b0 tx=0x7f807c003c20 comp rx=0 tx=0).stop 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.445+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 shutdown_connections 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.445+0000 7f808a3b2640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80841050b0 0x7f80841a16c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.446+0000 7f808a3b2640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f806c0afaf0 0x7f806c0b2000 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.446+0000 7f808a3b2640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8084102260 0x7f808419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.446+0000 7f808a3b2640 1 --2- 172.21.15.96:0/3061337414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8084101760 0x7f808419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.446+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 >> 172.21.15.96:0/3061337414 conn(0x7f80840fd380 msgr2=0x7f8084107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.446+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 shutdown_connections 2024-04-25T17:09:39.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.446+0000 7f808a3b2640 1 -- 172.21.15.96:0/3061337414 wait complete. 2024-04-25T17:09:39.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T17:09:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:39 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1502648283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:39.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:39 smithi097 ceph-mon[32602]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:39.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:39 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1781914558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:09:39.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:39 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1502648283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:39.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:39 smithi171 ceph-mon[33189]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:39.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:39 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1781914558' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 -- 172.21.15.96:0/2699250762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f8102190 msgr2=0x7f18f8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 --2- 172.21.15.96:0/2699250762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f8102190 0x7f18f8102e90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18e800aa10 tx=0x7f18e802b720 comp rx=0 tx=0).stop 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 -- 172.21.15.96:0/2699250762 shutdown_connections 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 --2- 172.21.15.96:0/2699250762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18f81080d0 0x7f18f810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 --2- 172.21.15.96:0/2699250762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18f81033d0 0x7f18f81037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 --2- 172.21.15.96:0/2699250762 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f8102190 0x7f18f8102e90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 -- 172.21.15.96:0/2699250762 >> 172.21.15.96:0/2699250762 conn(0x7f18f80fd380 msgr2=0x7f18f80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 -- 172.21.15.96:0/2699250762 shutdown_connections 2024-04-25T17:09:39.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.567+0000 7f18fe906640 1 -- 172.21.15.96:0/2699250762 wait complete. 2024-04-25T17:09:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 Processor -- start 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 -- start start 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18f8102190 0x7f18f8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f81033d0 0x7f18f81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18f81080d0 0x7f18f819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810d420 con 0x7f18f81033d0 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810d2a0 con 0x7f18f81080d0 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fe906640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810d5a0 con 0x7f18f8102190 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f81033d0 0x7f18f81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f81033d0 0x7f18f81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36480/0 (socket says 172.21.15.96:36480) 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 -- 172.21.15.96:0/3054502859 learned_addr learned my addr 172.21.15.96:0/3054502859 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f7fff640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18f8102190 0x7f18f8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18fcea3640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18f81080d0 0x7f18f819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 -- 172.21.15.96:0/3054502859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18f8102190 msgr2=0x7f18f8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18f8102190 0x7f18f8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 -- 172.21.15.96:0/3054502859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18f81080d0 msgr2=0x7f18f819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18f81080d0 0x7f18f819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.568+0000 7f18f77fe640 1 -- 172.21.15.96:0/3054502859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f819fb00 con 0x7f18f81033d0 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.569+0000 7f18f77fe640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f81033d0 0x7f18f81985b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e9f0 tx=0x7f18ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.569+0000 7f18f57fa640 1 -- 172.21.15.96:0/3054502859 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00cd80 con 0x7f18f81033d0 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.569+0000 7f18f57fa640 1 -- 172.21.15.96:0/3054502859 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ec0054d0 con 0x7f18f81033d0 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.569+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f819fdf0 con 0x7f18f81033d0 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.569+0000 7f18f57fa640 1 -- 172.21.15.96:0/3054502859 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec010690 con 0x7f18f81033d0 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.569+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f81a0250 con 0x7f18f81033d0 2024-04-25T17:09:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.570+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f8106ee0 con 0x7f18f81033d0 2024-04-25T17:09:39.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.571+0000 7f18f57fa640 1 -- 172.21.15.96:0/3054502859 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f18ec004e30 con 0x7f18f81033d0 2024-04-25T17:09:39.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.572+0000 7f18f57fa640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f18e00afb70 0x7f18e00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.572+0000 7f18f7fff640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f18e00afb70 0x7f18e00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.572+0000 7f18f57fa640 1 -- 172.21.15.96:0/3054502859 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f18ec0df1d0 con 0x7f18f81033d0 2024-04-25T17:09:39.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.572+0000 7f18f7fff640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f18e00afb70 0x7f18e00b2080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005ab0 tx=0x7f18e8030510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.576+0000 7f18f57fa640 1 -- 172.21.15.96:0/3054502859 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18ec053ff0 con 0x7f18f81033d0 2024-04-25T17:09:39.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.752+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18f80637d0 con 0x7f18f81033d0 2024-04-25T17:09:39.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.752+0000 7f18f57fa640 1 -- 172.21.15.96:0/3054502859 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f18ec0a7db0 con 0x7f18f81033d0 2024-04-25T17:09:39.752 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T17:09:39.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f18e00afb70 msgr2=0x7f18e00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f18e00afb70 0x7f18e00b2080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005ab0 tx=0x7f18e8030510 comp rx=0 tx=0).stop 2024-04-25T17:09:39.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f81033d0 msgr2=0x7f18f81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f81033d0 0x7f18f81985b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00e9f0 tx=0x7f18ec00edf0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 shutdown_connections 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18f81080d0 0x7f18f819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18f81033d0 0x7f18f81985b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f18e00afb70 0x7f18e00b2080 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 --2- 172.21.15.96:0/3054502859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f18f8102190 0x7f18f8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 >> 172.21.15.96:0/3054502859 conn(0x7f18f80fd380 msgr2=0x7f18f81040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 shutdown_connections 2024-04-25T17:09:39.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.754+0000 7f18fe906640 1 -- 172.21.15.96:0/3054502859 wait complete. 2024-04-25T17:09:39.765 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:09:39.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 -- 172.21.15.96:0/2660313134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 msgr2=0x7effbc103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 --2- 172.21.15.96:0/2660313134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc103fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effb0009900 tx=0x7effb002a410 comp rx=0 tx=0).stop 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 -- 172.21.15.96:0/2660313134 shutdown_connections 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 --2- 172.21.15.96:0/2660313134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effbc104670 0x7effbc10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 --2- 172.21.15.96:0/2660313134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc103fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 --2- 172.21.15.96:0/2660313134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effbc102fb0 0x7effbc1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 -- 172.21.15.96:0/2660313134 >> 172.21.15.96:0/2660313134 conn(0x7effbc0fd330 msgr2=0x7effbc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 -- 172.21.15.96:0/2660313134 shutdown_connections 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.873+0000 7effc1e61640 1 -- 172.21.15.96:0/2660313134 wait complete. 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 Processor -- start 2024-04-25T17:09:39.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 -- start start 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effbc102fb0 0x7effbc10d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc10bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effbc10c0c0 0x7effbc10c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effbc10f8a0 con 0x7effbc103b10 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7effbc10f720 con 0x7effbc102fb0 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effc1e61640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7effbc10fa20 con 0x7effbc10c0c0 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effbaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc10bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effbaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc10bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36498/0 (socket says 172.21.15.96:36498) 2024-04-25T17:09:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effbaffd640 1 -- 172.21.15.96:0/3101159795 learned_addr learned my addr 172.21.15.96:0/3101159795 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.874+0000 7effbbfff640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effbc10c0c0 0x7effbc10c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbb7fe640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effbc102fb0 0x7effbc10d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbaffd640 1 -- 172.21.15.96:0/3101159795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effbc10c0c0 msgr2=0x7effbc10c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbaffd640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effbc10c0c0 0x7effbc10c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbaffd640 1 -- 172.21.15.96:0/3101159795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effbc102fb0 msgr2=0x7effbc10d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbaffd640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effbc102fb0 0x7effbc10d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbaffd640 1 -- 172.21.15.96:0/3101159795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc1a4240 con 0x7effbc103b10 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbbfff640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effbc10c0c0 0x7effbc10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effbaffd640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc10bb80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7effb0000c00 tx=0x7effb00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effb8ff9640 1 -- 172.21.15.96:0/3101159795 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb003f030 con 0x7effbc103b10 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effb8ff9640 1 -- 172.21.15.96:0/3101159795 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb0003cb0 con 0x7effbc103b10 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc1a44d0 con 0x7effbc103b10 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effb8ff9640 1 -- 172.21.15.96:0/3101159795 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0035070 con 0x7effbc103b10 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.875+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc1a4a40 con 0x7effbc103b10 2024-04-25T17:09:39.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.876+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc069c70 con 0x7effbc103b10 2024-04-25T17:09:39.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.877+0000 7effb8ff9640 1 -- 172.21.15.96:0/3101159795 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7effb0042030 con 0x7effbc103b10 2024-04-25T17:09:39.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.878+0000 7effb8ff9640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7effa40afaf0 0x7effa40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:39.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.878+0000 7effbb7fe640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7effa40afaf0 0x7effa40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:39.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.878+0000 7effb8ff9640 1 -- 172.21.15.96:0/3101159795 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7effb00ff850 con 0x7effbc103b10 2024-04-25T17:09:39.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.879+0000 7effbb7fe640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7effa40afaf0 0x7effa40b2000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7effbc103a10 tx=0x7effac008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:39.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:39.882+0000 7effb8ff9640 1 -- 172.21.15.96:0/3101159795 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effb00c8430 con 0x7effbc103b10 2024-04-25T17:09:40.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.095+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effbc1073c0 con 0x7effbc103b10 2024-04-25T17:09:40.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.096+0000 7effb8ff9640 1 -- 172.21.15.96:0/3101159795 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11805 (secure 0 0 0) 0x7effb00c7d30 con 0x7effbc103b10 2024-04-25T17:09:40.095 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:40.096 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:08:14.735284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:09:40.096 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7effa40afaf0 msgr2=0x7effa40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7effa40afaf0 0x7effa40b2000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7effbc103a10 tx=0x7effac008040 comp rx=0 tx=0).stop 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 msgr2=0x7effbc10bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc10bb80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7effb0000c00 tx=0x7effb00043b0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 shutdown_connections 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7effa40afaf0 0x7effa40b2000 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effbc10c0c0 0x7effbc10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effbc103b10 0x7effbc10bb80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 --2- 172.21.15.96:0/3101159795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effbc102fb0 0x7effbc10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 >> 172.21.15.96:0/3101159795 conn(0x7effbc0fd330 msgr2=0x7effbc0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 shutdown_connections 2024-04-25T17:09:40.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.098+0000 7effc1e61640 1 -- 172.21.15.96:0/3101159795 wait complete. 2024-04-25T17:09:40.110 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.220+0000 7fdf6759e640 1 -- 172.21.15.96:0/2554851661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68101560 msgr2=0x7fdf681019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 --2- 172.21.15.96:0/2554851661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68101560 0x7fdf681019d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009900 tx=0x7fdf5c02a410 comp rx=0 tx=0).stop 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 -- 172.21.15.96:0/2554851661 shutdown_connections 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 --2- 172.21.15.96:0/2554851661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf68102060 0x7fdf6810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 --2- 172.21.15.96:0/2554851661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68101560 0x7fdf681019d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 --2- 172.21.15.96:0/2554851661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf68104eb0 0x7fdf681052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 -- 172.21.15.96:0/2554851661 >> 172.21.15.96:0/2554851661 conn(0x7fdf680fd1c0 msgr2=0x7fdf680ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 -- 172.21.15.96:0/2554851661 shutdown_connections 2024-04-25T17:09:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 -- 172.21.15.96:0/2554851661 wait complete. 2024-04-25T17:09:40.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 Processor -- start 2024-04-25T17:09:40.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.221+0000 7fdf6759e640 1 -- start start 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6759e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf68101560 0x7fdf6819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf68102060 0x7fdf6819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68104eb0 0x7fdf681a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6759e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6810f440 con 0x7fdf68104eb0 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6759e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6810f2c0 con 0x7fdf68102060 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6759e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6810f5c0 con 0x7fdf68101560 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68104eb0 0x7fdf681a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68104eb0 0x7fdf681a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36520/0 (socket says 172.21.15.96:36520) 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 -- 172.21.15.96:0/1915440097 learned_addr learned my addr 172.21.15.96:0/1915440097 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf65d9b640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf68102060 0x7fdf6819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6659c640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf68101560 0x7fdf6819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 -- 172.21.15.96:0/1915440097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf68101560 msgr2=0x7fdf6819a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf68101560 0x7fdf6819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 -- 172.21.15.96:0/1915440097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf68102060 msgr2=0x7fdf6819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf68102060 0x7fdf6819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 -- 172.21.15.96:0/1915440097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf681a1b80 con 0x7fdf68104eb0 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf66d9d640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68104eb0 0x7fdf681a1510 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c970 tx=0x7fdf6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf577fe640 1 -- 172.21.15.96:0/1915440097 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60007bb0 con 0x7fdf68104eb0 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf577fe640 1 -- 172.21.15.96:0/1915440097 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf60007d50 con 0x7fdf68104eb0 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.222+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf681a1e70 con 0x7fdf68104eb0 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.223+0000 7fdf577fe640 1 -- 172.21.15.96:0/1915440097 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60004e60 con 0x7fdf68104eb0 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.223+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681a22d0 con 0x7fdf68104eb0 2024-04-25T17:09:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.223+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf680691b0 con 0x7fdf68104eb0 2024-04-25T17:09:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.225+0000 7fdf577fe640 1 -- 172.21.15.96:0/1915440097 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdf60005000 con 0x7fdf68104eb0 2024-04-25T17:09:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.225+0000 7fdf577fe640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf500afae0 0x7fdf500b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.225+0000 7fdf6659c640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf500afae0 0x7fdf500b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.226+0000 7fdf577fe640 1 -- 172.21.15.96:0/1915440097 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fdf600debe0 con 0x7fdf68104eb0 2024-04-25T17:09:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.226+0000 7fdf6659c640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf500afae0 0x7fdf500b1ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf58005c40 tx=0x7fdf58005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:40.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.229+0000 7fdf577fe640 1 -- 172.21.15.96:0/1915440097 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf60014030 con 0x7fdf68104eb0 2024-04-25T17:09:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.409+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf68108890 con 0x7fdf68104eb0 2024-04-25T17:09:40.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.410+0000 7fdf577fe640 1 -- 172.21.15.96:0/1915440097 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22594 (secure 0 0 0) 0x7fdf68108890 con 0x7fdf68104eb0 2024-04-25T17:09:40.411 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:40.411 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:09:04.141423+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf500afae0 msgr2=0x7fdf500b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf500afae0 0x7fdf500b1ff0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf58005c40 tx=0x7fdf58005bd0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68104eb0 msgr2=0x7fdf681a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68104eb0 0x7fdf681a1510 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c970 tx=0x7fdf6000ce30 comp rx=0 tx=0).stop 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 shutdown_connections 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf68104eb0 0x7fdf681a1510 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf68102060 0x7fdf6819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf500afae0 0x7fdf500b1ff0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 --2- 172.21.15.96:0/1915440097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf68101560 0x7fdf6819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 >> 172.21.15.96:0/1915440097 conn(0x7fdf680fd1c0 msgr2=0x7fdf68105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:40.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 shutdown_connections 2024-04-25T17:09:40.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.413+0000 7fdf6759e640 1 -- 172.21.15.96:0/1915440097 wait complete. 2024-04-25T17:09:40.423 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:40 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3061337414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:09:40.423 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:40 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3054502859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:09:40.423 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:40 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3101159795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:40.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T17:08:09.266201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T17:08:10.452964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:09:40.425 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:09:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:40 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3061337414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:09:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:40 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3054502859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:09:40.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:40 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3101159795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:40.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:40 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3061337414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:09:40.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:40 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3054502859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:09:40.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:40 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3101159795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:40.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 -- 172.21.15.96:0/3599894459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc102180 msgr2=0x7f6bdc102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 --2- 172.21.15.96:0/3599894459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc102180 0x7f6bdc102e80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009900 tx=0x7f6bd402a410 comp rx=0 tx=0).stop 2024-04-25T17:09:40.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 -- 172.21.15.96:0/3599894459 shutdown_connections 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 --2- 172.21.15.96:0/3599894459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bdc1080c0 0x7f6bdc10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 --2- 172.21.15.96:0/3599894459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1033c0 0x7f6bdc1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 --2- 172.21.15.96:0/3599894459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc102180 0x7f6bdc102e80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 -- 172.21.15.96:0/3599894459 >> 172.21.15.96:0/3599894459 conn(0x7f6bdc0fd350 msgr2=0x7f6bdc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 -- 172.21.15.96:0/3599894459 shutdown_connections 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.536+0000 7f6be13e3640 1 -- 172.21.15.96:0/3599894459 wait complete. 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 Processor -- start 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 -- start start 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bdc102180 0x7f6bdc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1033c0 0x7f6bdc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc1080c0 0x7f6bdc1a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc10f620 con 0x7f6bdc1080c0 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc10f4a0 con 0x7f6bdc102180 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6be13e3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc10f7a0 con 0x7f6bdc1033c0 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc1080c0 0x7f6bdc1a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc1080c0 0x7f6bdc1a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36538/0 (socket says 172.21.15.96:36538) 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 -- 172.21.15.96:0/1156180602 learned_addr learned my addr 172.21.15.96:0/1156180602 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bda7fc640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1033c0 0x7f6bdc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdaffd640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bdc102180 0x7f6bdc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 -- 172.21.15.96:0/1156180602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1033c0 msgr2=0x7f6bdc19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1033c0 0x7f6bdc19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 -- 172.21.15.96:0/1156180602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bdc102180 msgr2=0x7f6bdc19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bdc102180 0x7f6bdc19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bdb7fe640 1 -- 172.21.15.96:0/1156180602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc1a1db0 con 0x7f6bdc1080c0 2024-04-25T17:09:40.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.537+0000 7f6bda7fc640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1033c0 0x7f6bdc19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:40.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.538+0000 7f6bdb7fe640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc1080c0 0x7f6bdc1a1740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009fd0 tx=0x7f6bcc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:40.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.538+0000 7f6bc3fff640 1 -- 172.21.15.96:0/1156180602 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc017070 con 0x7f6bdc1080c0 2024-04-25T17:09:40.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.538+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc1a20a0 con 0x7f6bdc1080c0 2024-04-25T17:09:40.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.538+0000 7f6bc3fff640 1 -- 172.21.15.96:0/1156180602 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bcc003e20 con 0x7f6bdc1080c0 2024-04-25T17:09:40.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.538+0000 7f6bc3fff640 1 -- 172.21.15.96:0/1156180602 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc010460 con 0x7f6bdc1080c0 2024-04-25T17:09:40.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.538+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a25a0 con 0x7f6bdc1080c0 2024-04-25T17:09:40.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.539+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bdc106ed0 con 0x7f6bdc1080c0 2024-04-25T17:09:40.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.540+0000 7f6bc3fff640 1 -- 172.21.15.96:0/1156180602 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6bcc003970 con 0x7f6bdc1080c0 2024-04-25T17:09:40.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.541+0000 7f6bc3fff640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6bc40afaf0 0x7f6bc40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.541+0000 7f6bdaffd640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6bc40afaf0 0x7f6bc40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.541+0000 7f6bc3fff640 1 -- 172.21.15.96:0/1156180602 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f6bcc0dea50 con 0x7f6bdc1080c0 2024-04-25T17:09:40.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.542+0000 7f6bdaffd640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6bc40afaf0 0x7f6bc40b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bd402a930 tx=0x7f6bd4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:40.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.545+0000 7f6bc3fff640 1 -- 172.21.15.96:0/1156180602 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bcc0a75f0 con 0x7f6bdc1080c0 2024-04-25T17:09:40.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.723+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6bdc0637a0 con 0x7f6bdc1080c0 2024-04-25T17:09:40.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.724+0000 7f6bc3fff640 1 -- 172.21.15.96:0/1156180602 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f6bcc0a6ef0 con 0x7f6bdc1080c0 2024-04-25T17:09:40.724 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:09:40.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6bc40afaf0 msgr2=0x7f6bc40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6bc40afaf0 0x7f6bc40b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bd402a930 tx=0x7f6bd4005fd0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc1080c0 msgr2=0x7f6bdc1a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc1080c0 0x7f6bdc1a1740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009fd0 tx=0x7f6bcc00b260 comp rx=0 tx=0).stop 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 shutdown_connections 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bdc1080c0 0x7f6bdc1a1740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f6bc40afaf0 0x7f6bc40b2000 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6bdc1033c0 0x7f6bdc19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 --2- 172.21.15.96:0/1156180602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bdc102180 0x7f6bdc19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 >> 172.21.15.96:0/1156180602 conn(0x7f6bdc0fd350 msgr2=0x7f6bdc1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 shutdown_connections 2024-04-25T17:09:40.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.726+0000 7f6be13e3640 1 -- 172.21.15.96:0/1156180602 wait complete. 2024-04-25T17:09:40.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T17:09:40.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 -- 172.21.15.96:0/896708022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c108940 msgr2=0x7f496c10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 --2- 172.21.15.96:0/896708022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c108940 0x7f496c10ad60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4964009ca0 tx=0x7f496402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 -- 172.21.15.96:0/896708022 shutdown_connections 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 --2- 172.21.15.96:0/896708022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c108940 0x7f496c10ad60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 --2- 172.21.15.96:0/896708022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f496c103c40 0x7f496c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 --2- 172.21.15.96:0/896708022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f496c102a00 0x7f496c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 -- 172.21.15.96:0/896708022 >> 172.21.15.96:0/896708022 conn(0x7f496c0fdbf0 msgr2=0x7f496c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 -- 172.21.15.96:0/896708022 shutdown_connections 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.848+0000 7f4974714640 1 -- 172.21.15.96:0/896708022 wait complete. 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 Processor -- start 2024-04-25T17:09:40.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 -- start start 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f496c102a00 0x7f496c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c103c40 0x7f496c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f496c108940 0x7f496c1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f496c10e530 con 0x7f496c103c40 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f496c10e3b0 con 0x7f496c108940 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4974714640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f496c10e6b0 con 0x7f496c102a00 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4971caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c103c40 0x7f496c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4971caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c103c40 0x7f496c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36552/0 (socket says 172.21.15.96:36552) 2024-04-25T17:09:40.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4971caf640 1 -- 172.21.15.96:0/1264746945 learned_addr learned my addr 172.21.15.96:0/1264746945 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f49724b0640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f496c102a00 0x7f496c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.849+0000 7f4972cb1640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f496c108940 0x7f496c1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4971caf640 1 -- 172.21.15.96:0/1264746945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f496c102a00 msgr2=0x7f496c1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4971caf640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f496c102a00 0x7f496c1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4971caf640 1 -- 172.21.15.96:0/1264746945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f496c108940 msgr2=0x7f496c1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4971caf640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f496c108940 0x7f496c1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4971caf640 1 -- 172.21.15.96:0/1264746945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c1a0bd0 con 0x7f496c103c40 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4972cb1640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f496c108940 0x7f496c1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4971caf640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c103c40 0x7f496c199620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f496800d770 tx=0x7f496800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f49637fe640 1 -- 172.21.15.96:0/1264746945 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968017070 con 0x7f496c103c40 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f49637fe640 1 -- 172.21.15.96:0/1264746945 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49680054d0 con 0x7f496c103c40 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c1a0ec0 con 0x7f496c103c40 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f49637fe640 1 -- 172.21.15.96:0/1264746945 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968004c30 con 0x7f496c103c40 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.850+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c1a1330 con 0x7f496c103c40 2024-04-25T17:09:40.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.851+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496c107750 con 0x7f496c103c40 2024-04-25T17:09:40.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.853+0000 7f49637fe640 1 -- 172.21.15.96:0/1264746945 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4968004dd0 con 0x7f496c103c40 2024-04-25T17:09:40.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.854+0000 7f49637fe640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f49580afaf0 0x7f49580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:40.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.854+0000 7f49724b0640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f49580afaf0 0x7f49580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:40.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.854+0000 7f49637fe640 1 -- 172.21.15.96:0/1264746945 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f49680deaf0 con 0x7f496c103c40 2024-04-25T17:09:40.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.854+0000 7f49724b0640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f49580afaf0 0x7f49580b2000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f496c103600 tx=0x7f495c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:40.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:40.857+0000 7f49637fe640 1 -- 172.21.15.96:0/1264746945 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49680a76d0 con 0x7f496c103c40 2024-04-25T17:09:41.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.036+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f496c02dce0 con 0x7f496c103c40 2024-04-25T17:09:41.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.037+0000 7f49637fe640 1 -- 172.21.15.96:0/1264746945 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f49680a6fd0 con 0x7f496c103c40 2024-04-25T17:09:41.036 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.038+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f49580afaf0 msgr2=0x7f49580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.038+0000 7f4974714640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f49580afaf0 0x7f49580b2000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f496c103600 tx=0x7f495c005f90 comp rx=0 tx=0).stop 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.038+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c103c40 msgr2=0x7f496c199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.038+0000 7f4974714640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c103c40 0x7f496c199620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f496800d770 tx=0x7f496800dc30 comp rx=0 tx=0).stop 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 shutdown_connections 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f496c108940 0x7f496c1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f496c103c40 0x7f496c199620 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f49580afaf0 0x7f49580b2000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 --2- 172.21.15.96:0/1264746945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f496c102a00 0x7f496c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 >> 172.21.15.96:0/1264746945 conn(0x7f496c0fdbf0 msgr2=0x7f496c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 shutdown_connections 2024-04-25T17:09:41.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.039+0000 7f4974714640 1 -- 172.21.15.96:0/1264746945 wait complete. 2024-04-25T17:09:41.050 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T17:09:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 -- 172.21.15.96:0/3977494243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 msgr2=0x7f6770101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 --2- 172.21.15.96:0/3977494243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f6770101bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6758009900 tx=0x7f675802a410 comp rx=0 tx=0).stop 2024-04-25T17:09:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 -- 172.21.15.96:0/3977494243 shutdown_connections 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 --2- 172.21.15.96:0/3977494243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6770102260 0x7f677010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 --2- 172.21.15.96:0/3977494243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f6770101bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 --2- 172.21.15.96:0/3977494243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67701050b0 0x7f67701054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 -- 172.21.15.96:0/3977494243 >> 172.21.15.96:0/3977494243 conn(0x7f67700fd380 msgr2=0x7f67700ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 -- 172.21.15.96:0/3977494243 shutdown_connections 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 -- 172.21.15.96:0/3977494243 wait complete. 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 Processor -- start 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.161+0000 7f6775c30640 1 -- start start 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f677019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6770102260 0x7f677019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67701050b0 0x7f67701a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f677010f640 con 0x7f6770101760 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f677010f4c0 con 0x7f67701050b0 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f677010f7c0 con 0x7f6770102260 2024-04-25T17:09:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f677019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f677019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36568/0 (socket says 172.21.15.96:36568) 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 -- 172.21.15.96:0/1453470525 learned_addr learned my addr 172.21.15.96:0/1453470525 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6767fff640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6770102260 0x7f677019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 -- 172.21.15.96:0/1453470525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6770102260 msgr2=0x7f677019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6770102260 0x7f677019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 -- 172.21.15.96:0/1453470525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67701050b0 msgr2=0x7f67701a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67701050b0 0x7f67701a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 -- 172.21.15.96:0/1453470525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67701a1d00 con 0x7f6770101760 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6767fff640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6770102260 0x7f677019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676f7fe640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f677019a270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f676000c970 tx=0x7f676000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676d7fa640 1 -- 172.21.15.96:0/1453470525 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760004220 con 0x7f6770101760 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676d7fa640 1 -- 172.21.15.96:0/1453470525 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67600043c0 con 0x7f6770101760 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67701a1ff0 con 0x7f6770101760 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f676d7fa640 1 -- 172.21.15.96:0/1453470525 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760004a00 con 0x7f6770101760 2024-04-25T17:09:41.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.162+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67701a2400 con 0x7f6770101760 2024-04-25T17:09:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.163+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677006a160 con 0x7f6770101760 2024-04-25T17:09:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.165+0000 7f676d7fa640 1 -- 172.21.15.96:0/1453470525 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f67600079b0 con 0x7f6770101760 2024-04-25T17:09:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.166+0000 7f676d7fa640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f675c0afaf0 0x7f675c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.166+0000 7f6767fff640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f675c0afaf0 0x7f675c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.166+0000 7f676d7fa640 1 -- 172.21.15.96:0/1453470525 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f67600deeb0 con 0x7f6770101760 2024-04-25T17:09:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.166+0000 7f6767fff640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f675c0afaf0 0x7f675c0b2000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6758006010 tx=0x7f6758000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.170+0000 7f676d7fa640 1 -- 172.21.15.96:0/1453470525 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6760014030 con 0x7f6770101760 2024-04-25T17:09:41.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.349+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6770108a90 con 0x7f6770101760 2024-04-25T17:09:41.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.349+0000 7f676d7fa640 1 -- 172.21.15.96:0/1453470525 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f6770108a90 con 0x7f6770101760 2024-04-25T17:09:41.349 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T17:09:41.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f675c0afaf0 msgr2=0x7f675c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f675c0afaf0 0x7f675c0b2000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6758006010 tx=0x7f6758000bc0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 msgr2=0x7f677019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f677019a270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f676000c970 tx=0x7f676000ce30 comp rx=0 tx=0).stop 2024-04-25T17:09:41.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 shutdown_connections 2024-04-25T17:09:41.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67701050b0 0x7f67701a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f675c0afaf0 0x7f675c0b2000 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6770102260 0x7f677019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 --2- 172.21.15.96:0/1453470525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6770101760 0x7f677019a270 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 >> 172.21.15.96:0/1453470525 conn(0x7f67700fd380 msgr2=0x7f6770105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:41.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 shutdown_connections 2024-04-25T17:09:41.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.351+0000 7f6775c30640 1 -- 172.21.15.96:0/1453470525 wait complete. 2024-04-25T17:09:41.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:41 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1915440097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:41.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:41 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1156180602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:09:41.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:41 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1264746945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:09:41.362 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:41 smithi096 ceph-mon[29131]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:41.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:09:41.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.469+0000 7efd684ca640 1 -- 172.21.15.96:0/779774665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 msgr2=0x7efd601090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.469+0000 7efd684ca640 1 --2- 172.21.15.96:0/779774665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd601090c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efd54009900 tx=0x7efd5402a410 comp rx=0 tx=0).stop 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- 172.21.15.96:0/779774665 shutdown_connections 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 --2- 172.21.15.96:0/779774665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd60109750 0x7efd6010bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 --2- 172.21.15.96:0/779774665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd601090c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 --2- 172.21.15.96:0/779774665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd601041f0 0x7efd60106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- 172.21.15.96:0/779774665 >> 172.21.15.96:0/779774665 conn(0x7efd600fdbd0 msgr2=0x7efd60100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- 172.21.15.96:0/779774665 shutdown_connections 2024-04-25T17:09:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- 172.21.15.96:0/779774665 wait complete. 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 Processor -- start 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- start start 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd601041f0 0x7efd6019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd6019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd60109750 0x7efd601a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd6010f5c0 con 0x7efd60106be0 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd6010f440 con 0x7efd601041f0 2024-04-25T17:09:41.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.470+0000 7efd684ca640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efd6010f740 con 0x7efd60109750 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd6019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd6019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36588/0 (socket says 172.21.15.96:36588) 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 -- 172.21.15.96:0/181550767 learned_addr learned my addr 172.21.15.96:0/181550767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd66266640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd601041f0 0x7efd6019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 -- 172.21.15.96:0/181550767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd60109750 msgr2=0x7efd601a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd60109750 0x7efd601a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 -- 172.21.15.96:0/181550767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd601041f0 msgr2=0x7efd6019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd601041f0 0x7efd6019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 -- 172.21.15.96:0/181550767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd601a1d10 con 0x7efd60106be0 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd66266640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd601041f0 0x7efd6019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd65a65640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd6019a710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd54006010 tx=0x7efd54004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd5f7fe640 1 -- 172.21.15.96:0/181550767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5403f030 con 0x7efd60106be0 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd5f7fe640 1 -- 172.21.15.96:0/181550767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54003cb0 con 0x7efd60106be0 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd601a1fa0 con 0x7efd60106be0 2024-04-25T17:09:41.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd5f7fe640 1 -- 172.21.15.96:0/181550767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54035070 con 0x7efd60106be0 2024-04-25T17:09:41.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.471+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd601a2400 con 0x7efd60106be0 2024-04-25T17:09:41.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.472+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd600ff6f0 con 0x7efd60106be0 2024-04-25T17:09:41.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.474+0000 7efd5f7fe640 1 -- 172.21.15.96:0/181550767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7efd54042030 con 0x7efd60106be0 2024-04-25T17:09:41.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.474+0000 7efd5f7fe640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efd480afaf0 0x7efd480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.474+0000 7efd66266640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efd480afaf0 0x7efd480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.474+0000 7efd5f7fe640 1 -- 172.21.15.96:0/181550767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7efd540ff9c0 con 0x7efd60106be0 2024-04-25T17:09:41.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.475+0000 7efd66266640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efd480afaf0 0x7efd480b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd6006a850 tx=0x7efd50009400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:41.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.478+0000 7efd5f7fe640 1 -- 172.21.15.96:0/181550767 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd540c85a0 con 0x7efd60106be0 2024-04-25T17:09:41.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:41 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1915440097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:41.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:41 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1156180602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:09:41.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:41 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1264746945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:09:41.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:41 smithi097 ceph-mon[32602]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:41.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:41 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1915440097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:41.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:41 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1156180602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:09:41.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:41 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1264746945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:09:41.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:41 smithi171 ceph-mon[33189]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:41.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.690+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6002db50 con 0x7efd60106be0 2024-04-25T17:09:41.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.691+0000 7efd5f7fe640 1 -- 172.21.15.96:0/181550767 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11805 (secure 0 0 0) 0x7efd54003e60 con 0x7efd60106be0 2024-04-25T17:09:41.691 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:41.691 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:08:14.735284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:09:41.691 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-04-25T17:09:41.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efd480afaf0 msgr2=0x7efd480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efd480afaf0 0x7efd480b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efd6006a850 tx=0x7efd50009400 comp rx=0 tx=0).stop 2024-04-25T17:09:41.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 msgr2=0x7efd6019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd6019a710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efd54006010 tx=0x7efd54004340 comp rx=0 tx=0).stop 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 shutdown_connections 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7efd480afaf0 0x7efd480b2000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd60109750 0x7efd601a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd60106be0 0x7efd6019a710 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 --2- 172.21.15.96:0/181550767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd601041f0 0x7efd6019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 >> 172.21.15.96:0/181550767 conn(0x7efd600fdbd0 msgr2=0x7efd60104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 shutdown_connections 2024-04-25T17:09:41.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.693+0000 7efd684ca640 1 -- 172.21.15.96:0/181550767 wait complete. 2024-04-25T17:09:41.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 -- 172.21.15.96:0/3082105859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0108900 msgr2=0x7f11e010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 --2- 172.21.15.96:0/3082105859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0108900 0x7f11e010ad20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009ca0 tx=0x7f11d802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 -- 172.21.15.96:0/3082105859 shutdown_connections 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 --2- 172.21.15.96:0/3082105859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0108900 0x7f11e010ad20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 --2- 172.21.15.96:0/3082105859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11e0103c00 0x7f11e0104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 --2- 172.21.15.96:0/3082105859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11e01029c0 0x7f11e01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 -- 172.21.15.96:0/3082105859 >> 172.21.15.96:0/3082105859 conn(0x7f11e00fdbd0 msgr2=0x7f11e0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 -- 172.21.15.96:0/3082105859 shutdown_connections 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 -- 172.21.15.96:0/3082105859 wait complete. 2024-04-25T17:09:41.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 Processor -- start 2024-04-25T17:09:41.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.812+0000 7f11e6964640 1 -- start start 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11e01029c0 0x7f11e006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0103c00 0x7f11e019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11e0108900 0x7f11e01a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11e010e540 con 0x7f11e0103c00 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f11e010e3c0 con 0x7f11e01029c0 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f11e010e6c0 con 0x7f11e0108900 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0103c00 0x7f11e019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0103c00 0x7f11e019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36616/0 (socket says 172.21.15.96:36616) 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 -- 172.21.15.96:0/204575140 learned_addr learned my addr 172.21.15.96:0/204575140 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11dffff640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11e01029c0 0x7f11e006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e4f01640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11e0108900 0x7f11e01a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 -- 172.21.15.96:0/204575140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11e0108900 msgr2=0x7f11e01a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11e0108900 0x7f11e01a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 -- 172.21.15.96:0/204575140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11e01029c0 msgr2=0x7f11e006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11e01029c0 0x7f11e006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 -- 172.21.15.96:0/204575140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e01a1d00 con 0x7f11e0103c00 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11dffff640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11e01029c0 0x7f11e006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11df7fe640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0103c00 0x7f11e019ef90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d950 tx=0x7f11d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11dd7fa640 1 -- 172.21.15.96:0/204575140 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0017070 con 0x7f11e0103c00 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11dd7fa640 1 -- 172.21.15.96:0/204575140 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d0009c30 con 0x7f11e0103c00 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e01a1ff0 con 0x7f11e0103c00 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11dd7fa640 1 -- 172.21.15.96:0/204575140 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d000f620 con 0x7f11e0103c00 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.813+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01a2460 con 0x7f11e0103c00 2024-04-25T17:09:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.815+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e0107710 con 0x7f11e0103c00 2024-04-25T17:09:41.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.816+0000 7f11dd7fa640 1 -- 172.21.15.96:0/204575140 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f11d0010460 con 0x7f11e0103c00 2024-04-25T17:09:41.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.817+0000 7f11dd7fa640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f11cc0afaf0 0x7f11cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:41.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.817+0000 7f11dffff640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f11cc0afaf0 0x7f11cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:41.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.817+0000 7f11dd7fa640 1 -- 172.21.15.96:0/204575140 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f11d00de8b0 con 0x7f11e0103c00 2024-04-25T17:09:41.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.817+0000 7f11dffff640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f11cc0afaf0 0x7f11cc0b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11c80058c0 tx=0x7f11c8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:41.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.821+0000 7f11dd7fa640 1 -- 172.21.15.96:0/204575140 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11d00a7490 con 0x7f11e0103c00 2024-04-25T17:09:41.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.997+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11e002dda0 con 0x7f11e0103c00 2024-04-25T17:09:41.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:41.999+0000 7f11dd7fa640 1 -- 172.21.15.96:0/204575140 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22594 (secure 0 0 0) 0x7f11d0009de0 con 0x7f11e0103c00 2024-04-25T17:09:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:41.999 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:09:04.141423+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f11cc0afaf0 msgr2=0x7f11cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f11cc0afaf0 0x7f11cc0b2000 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11c80058c0 tx=0x7f11c8005850 comp rx=0 tx=0).stop 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0103c00 msgr2=0x7f11e019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0103c00 0x7f11e019ef90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d950 tx=0x7f11d000de10 comp rx=0 tx=0).stop 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 shutdown_connections 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f11cc0afaf0 0x7f11cc0b2000 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11e0108900 0x7f11e01a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e0103c00 0x7f11e019ef90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 --2- 172.21.15.96:0/204575140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11e01029c0 0x7f11e006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 >> 172.21.15.96:0/204575140 conn(0x7f11e00fdbd0 msgr2=0x7f11e01048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 shutdown_connections 2024-04-25T17:09:42.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.001+0000 7f11e6964640 1 -- 172.21.15.96:0/204575140 wait complete. 2024-04-25T17:09:42.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 -- 172.21.15.96:0/3950294878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 msgr2=0x7f17a8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 --2- 172.21.15.96:0/3950294878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a8104800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009ca0 tx=0x7f17a402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 -- 172.21.15.96:0/3950294878 shutdown_connections 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 --2- 172.21.15.96:0/3950294878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a8109a40 0x7f17a810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 --2- 172.21.15.96:0/3950294878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17a8104d40 0x7f17a8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 --2- 172.21.15.96:0/3950294878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a8104800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 -- 172.21.15.96:0/3950294878 >> 172.21.15.96:0/3950294878 conn(0x7f17a80fed10 msgr2=0x7f17a81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 -- 172.21.15.96:0/3950294878 shutdown_connections 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 -- 172.21.15.96:0/3950294878 wait complete. 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 Processor -- start 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.120+0000 7f17afa92640 1 -- start start 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a8104d40 0x7f17a819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17a8109a40 0x7f17a81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17a810f6e0 con 0x7f17a8103b00 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17a810f560 con 0x7f17a8109a40 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17a810f860 con 0x7f17a8104d40 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36640/0 (socket says 172.21.15.96:36640) 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 -- 172.21.15.96:0/2172688443 learned_addr learned my addr 172.21.15.96:0/2172688443 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad02d640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a8104d40 0x7f17a819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ae02f640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17a8109a40 0x7f17a81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 -- 172.21.15.96:0/2172688443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a8104d40 msgr2=0x7f17a819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a8104d40 0x7f17a819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 -- 172.21.15.96:0/2172688443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17a8109a40 msgr2=0x7f17a81a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17a8109a40 0x7f17a81a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 -- 172.21.15.96:0/2172688443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a81a1f40 con 0x7f17a8103b00 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad02d640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a8104d40 0x7f17a819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ae02f640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17a8109a40 0x7f17a81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17ad82e640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a819a400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f17a4033040 tx=0x7f17a4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f179effd640 1 -- 172.21.15.96:0/2172688443 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4003e20 con 0x7f17a8103b00 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a81a21d0 con 0x7f17a8103b00 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f179effd640 1 -- 172.21.15.96:0/2172688443 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a40044f0 con 0x7f17a8103b00 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f179effd640 1 -- 172.21.15.96:0/2172688443 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a402fec0 con 0x7f17a8103b00 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.121+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a81a27f0 con 0x7f17a8103b00 2024-04-25T17:09:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.122+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a8108850 con 0x7f17a8103b00 2024-04-25T17:09:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.124+0000 7f179effd640 1 -- 172.21.15.96:0/2172688443 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f17a402f660 con 0x7f17a8103b00 2024-04-25T17:09:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.124+0000 7f179effd640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17940afc90 0x7f17940b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.124+0000 7f17ad02d640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17940afc90 0x7f17940b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.124+0000 7f179effd640 1 -- 172.21.15.96:0/2172688443 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f17a41004d0 con 0x7f17a8103b00 2024-04-25T17:09:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.125+0000 7f17ad02d640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17940afc90 0x7f17940b21a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1798006fd0 tx=0x7f1798008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.128+0000 7f179effd640 1 -- 172.21.15.96:0/2172688443 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17a40c90b0 con 0x7f17a8103b00 2024-04-25T17:09:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.343+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17a802db50 con 0x7f17a8103b00 2024-04-25T17:09:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.344+0000 7f179effd640 1 -- 172.21.15.96:0/2172688443 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11805 (secure 0 0 0) 0x7f17a40c89b0 con 0x7f17a8103b00 2024-04-25T17:09:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:42.344 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:08:14.735284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:09:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-04-25T17:09:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17940afc90 msgr2=0x7f17940b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17940afc90 0x7f17940b21a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1798006fd0 tx=0x7f1798008040 comp rx=0 tx=0).stop 2024-04-25T17:09:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 msgr2=0x7f17a819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a819a400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f17a4033040 tx=0x7f17a4003970 comp rx=0 tx=0).stop 2024-04-25T17:09:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 shutdown_connections 2024-04-25T17:09:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17a8109a40 0x7f17a81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f17940afc90 0x7f17940b21a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a8104d40 0x7f17a819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 --2- 172.21.15.96:0/2172688443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17a8103b00 0x7f17a819a400 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 >> 172.21.15.96:0/2172688443 conn(0x7f17a80fed10 msgr2=0x7f17a8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 shutdown_connections 2024-04-25T17:09:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.346+0000 7f17afa92640 1 -- 172.21.15.96:0/2172688443 wait complete. 2024-04-25T17:09:42.356 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:42 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1453470525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:09:42.356 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:42 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/181550767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:42.356 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:42 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/204575140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:42.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T17:08:11.494613+0000', 'modified': '2024-04-25T17:08:14.735284+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/825277463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 825277463}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 825277463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6839/3298672156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3298672156}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3298672156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24521]} 2024-04-25T17:09:42.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T17:09:42.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T17:09:42.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 -- 172.21.15.96:0/3836123409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 msgr2=0x7f65f4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 --2- 172.21.15.96:0/3836123409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4104030 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009900 tx=0x7f65e802a410 comp rx=0 tx=0).stop 2024-04-25T17:09:42.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 -- 172.21.15.96:0/3836123409 shutdown_connections 2024-04-25T17:09:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 --2- 172.21.15.96:0/3836123409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4108920 0x7f65f410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 --2- 172.21.15.96:0/3836123409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4104030 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 --2- 172.21.15.96:0/3836123409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65f41029e0 0x7f65f41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 -- 172.21.15.96:0/3836123409 >> 172.21.15.96:0/3836123409 conn(0x7f65f40fdbd0 msgr2=0x7f65f4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 -- 172.21.15.96:0/3836123409 shutdown_connections 2024-04-25T17:09:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.466+0000 7f65fa834640 1 -- 172.21.15.96:0/3836123409 wait complete. 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 Processor -- start 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 -- start start 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f41029e0 0x7f65f4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65f4108920 0x7f65f41a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65f410e4e0 con 0x7f65f4103c20 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f65f410e360 con 0x7f65f4108920 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65fa834640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65f410e660 con 0x7f65f41029e0 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36654/0 (socket says 172.21.15.96:36654) 2024-04-25T17:09:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.467+0000 7f65f37fe640 1 -- 172.21.15.96:0/794763002 learned_addr learned my addr 172.21.15.96:0/794763002 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65f4108920 0x7f65f41a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f3fff640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f41029e0 0x7f65f4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f37fe640 1 -- 172.21.15.96:0/794763002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f41029e0 msgr2=0x7f65f4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f37fe640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f41029e0 0x7f65f4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f37fe640 1 -- 172.21.15.96:0/794763002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65f4108920 msgr2=0x7f65f41a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f37fe640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65f4108920 0x7f65f41a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f37fe640 1 -- 172.21.15.96:0/794763002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41a0ea0 con 0x7f65f4103c20 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f37fe640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4199950 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006010 tx=0x7f65e8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f17fa640 1 -- 172.21.15.96:0/794763002 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e803f030 con 0x7f65f4103c20 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f17fa640 1 -- 172.21.15.96:0/794763002 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e8003cb0 con 0x7f65f4103c20 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f41a1130 con 0x7f65f4103c20 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65f17fa640 1 -- 172.21.15.96:0/794763002 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8035070 con 0x7f65f4103c20 2024-04-25T17:09:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.468+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a1620 con 0x7f65f4103c20 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.470+0000 7f65f17fa640 1 -- 172.21.15.96:0/794763002 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f65e8042030 con 0x7f65f4103c20 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.470+0000 7f66017be740 1 Processor -- start 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f66017be740 1 -- start start 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f66017be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf25270 0x5560bc2c6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f66017be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f66017be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f66017be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560bc22b740 con 0x5560bbefc3b0 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f66017be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5560bc1c73a0 con 0x5560bbf25270 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f66017be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5560bc227700 con 0x5560bbf04590 2024-04-25T17:09:42.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36658/0 (socket says 172.21.15.96:36658) 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 -- 172.21.15.96:0/881566936 learned_addr learned my addr 172.21.15.96:0/881566936 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf25270 0x5560bc2c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f37fe640 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f17fa640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f65dc0afaf0 0x7f65dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 -- 172.21.15.96:0/881566936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 msgr2=0x5560bc2cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 -- 172.21.15.96:0/881566936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf25270 msgr2=0x5560bc2c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf25270 0x5560bc2c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 -- 172.21.15.96:0/881566936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560bc1e3750 con 0x5560bbefc3b0 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf25270 0x5560bc2c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f37fe640 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f65dc0afaf0 0x7f65dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f17fa640 1 -- 172.21.15.96:0/794763002 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f65e80ff3a0 con 0x7f65f4103c20 2024-04-25T17:09:42.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.471+0000 7f65f3fff640 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2c68f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f65e400cc00 tx=0x7f65e400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13084eaec0d1dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f65da7fc640 1 -- 172.21.15.96:0/881566936 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004220 con 0x5560bbefc3b0 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f65da7fc640 1 -- 172.21.15.96:0/881566936 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e40043c0 con 0x5560bbefc3b0 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f65da7fc640 1 -- 172.21.15.96:0/881566936 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004e00 con 0x5560bbefc3b0 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f66017be740 1 -- 172.21.15.96:0/881566936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 msgr2=0x5560bc2c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f65dc0afaf0 0x7f65dc0b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5560bc2c7540 tx=0x7f65ec004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f66017be740 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2c68f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f65e400cc00 tx=0x7f65e400e720 comp rx=0 tx=0).stop 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f66017be740 1 -- 172.21.15.96:0/881566936 shutdown_connections 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f66017be740 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f66017be740 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2c68f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f66017be740 1 --2- 172.21.15.96:0/881566936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf25270 0x5560bc2c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.472+0000 7f66017be740 1 -- 172.21.15.96:0/881566936 >> 172.21.15.96:0/881566936 conn(0x5560bbf193c0 msgr2=0x5560bc2c1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 -- 172.21.15.96:0/881566936 shutdown_connections 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 -- 172.21.15.96:0/881566936 wait complete. 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 -- start start 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 Processor -- start 2024-04-25T17:09:42.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf04590 0x5560bc2d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf25270 0x5560bc2d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2ce240 con 0x5560bbefc3b0 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5560bc1b2fe0 con 0x5560bbf04590 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2ce3c0 con 0x5560bbf25270 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f66017be740 10 client.0 _ll_register_callbacks cb 0x5560bc1c9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f65f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f65f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36674/0 (socket says 172.21.15.96:36674) 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.473+0000 7f65f3fff640 1 -- 172.21.15.96:0/3105158857 learned_addr learned my addr 172.21.15.96:0/3105158857 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf25270 0x5560bc2d7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f37fe640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf04590 0x5560bc2d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f3fff640 1 -- 172.21.15.96:0/3105158857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf25270 msgr2=0x5560bc2d7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f3fff640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf25270 0x5560bc2d7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f3fff640 1 -- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf04590 msgr2=0x5560bc2d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f3fff640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf04590 0x5560bc2d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f3fff640 1 -- 172.21.15.96:0/3105158857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560bbf3d2d0 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f37fe640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf04590 0x5560bc2d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65f3fff640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d0270 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f65e40057c0 tx=0x7f65e401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fd322a6bdfabc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65d37fe640 1 -- 172.21.15.96:0/3105158857 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400c710 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65d37fe640 1 -- 172.21.15.96:0/3105158857 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f65e401cd10 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e40043c0 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65d37fe640 1 -- 172.21.15.96:0/3105158857 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f65e401c070 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f66017be740 10 client.15108 fetch_fsmap learned FSMap version 18 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5560bc0076b0 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.474+0000 7f65d37fe640 1 -- 172.21.15.96:0/3105158857 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e40043c0 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f65d37fe640 1 -- 172.21.15.96:0/3105158857 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f65e401acb0 con 0x5560bbefc3b0 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f66017be740 10 client.15108 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f66017be740 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f66017be740 20 client.15108 populate_metadata read hostname 'smithi096' 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f66017be740 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbee3e40 0x5560bc2ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f66017be740 4 client.15108 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:09:42.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560bc239350 con 0x5560bbee3e40 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f65f37fe640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbee3e40 0x5560bc2ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f65d37fe640 10 client.15108 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.475+0000 7f65f37fe640 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbee3e40 0x5560bc2ddcd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5a23435019cfdd2 server_cookie=64dcc31b1fa8e682 in_seq=0 out_seq=0 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f65d37fe640 1 -- 172.21.15.96:0/3105158857 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560bc239350 con 0x5560bbee3e40 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f65d37fe640 10 client.15108 handle_command_reply: tid=0 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f66017be740 1 client.15108 shutdown 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 msgr2=0x5560bc2d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f66017be740 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d0270 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f65e40057c0 tx=0x7f65e401cc20 comp rx=0 tx=0).stop 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 shutdown_connections 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f66017be740 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbee3e40 0x5560bc2ddcd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f65d37fe640 0 client.15108 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.476+0000 7f66017be740 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf25270 0x5560bc2d7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.477+0000 7f66017be740 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbf04590 0x5560bc2d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.477+0000 7f66017be740 1 --2- 172.21.15.96:0/3105158857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d0270 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.477+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 >> 172.21.15.96:0/3105158857 conn(0x5560bbf193c0 msgr2=0x5560bc2c0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.477+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 shutdown_connections 2024-04-25T17:09:42.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.477+0000 7f66017be740 1 -- 172.21.15.96:0/3105158857 wait complete. 2024-04-25T17:09:42.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.477+0000 7f66017be740 20 client.15108 trim_cache size 0 max 16384 2024-04-25T17:09:42.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:42 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1453470525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:09:42.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:42 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/181550767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:42.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:42 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/204575140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:42.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 Processor -- start 2024-04-25T17:09:42.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 -- start start 2024-04-25T17:09:42.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbefc3b0 0x5560bc2cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbf04590 0x5560bc2c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2dc010 con 0x5560bbf04590 2024-04-25T17:09:42.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2dc6e0 con 0x5560bbee3e40 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f66017be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2dcdb0 con 0x5560bbefc3b0 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f8dd1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbf04590 0x5560bc2c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f8dd1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbf04590 0x5560bc2c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36678/0 (socket says 172.21.15.96:36678) 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f8dd1640 1 -- 172.21.15.96:0/2884994273 learned_addr learned my addr 172.21.15.96:0/2884994273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f3fff640 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f37fe640 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbefc3b0 0x5560bc2cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f8dd1640 1 -- 172.21.15.96:0/2884994273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbefc3b0 msgr2=0x5560bc2cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbefc3b0 0x5560bc2cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f8dd1640 1 -- 172.21.15.96:0/2884994273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 msgr2=0x5560bc2ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.495+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f65f8dd1640 1 -- 172.21.15.96:0/2884994273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560bc2c70d0 con 0x5560bbf04590 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f65f37fe640 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbefc3b0 0x5560bc2cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f65f8dd1640 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbf04590 0x5560bc2c16b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f65ec012770 tx=0x7f65ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4164ac956dbce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f65d9ffb640 1 -- 172.21.15.96:0/2884994273 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec01e8c0 con 0x5560bbf04590 2024-04-25T17:09:42.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f65d9ffb640 1 -- 172.21.15.96:0/2884994273 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ec016070 con 0x5560bbf04590 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f65d9ffb640 1 -- 172.21.15.96:0/2884994273 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0263e0 con 0x5560bbf04590 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f66017be740 1 -- 172.21.15.96:0/2884994273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbf04590 msgr2=0x5560bc2c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.496+0000 7f66017be740 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbf04590 0x5560bc2c16b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f65ec012770 tx=0x7f65ec014000 comp rx=0 tx=0).stop 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- 172.21.15.96:0/2884994273 shutdown_connections 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbf04590 0x5560bc2c16b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbefc3b0 0x5560bc2cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 --2- 172.21.15.96:0/2884994273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- 172.21.15.96:0/2884994273 >> 172.21.15.96:0/2884994273 conn(0x5560bbf193c0 msgr2=0x5560bc2c9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- 172.21.15.96:0/2884994273 shutdown_connections 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- 172.21.15.96:0/2884994273 wait complete. 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- start start 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 Processor -- start 2024-04-25T17:09:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2ddb50 con 0x5560bbefc3b0 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2dd480 con 0x5560bbee3e40 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5560bc2de220 con 0x5560bbf04590 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f66017be740 10 client.0 _ll_register_callbacks cb 0x7f65e4014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f65f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f65f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36684/0 (socket says 172.21.15.96:36684) 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.497+0000 7f65f37fe640 1 -- 172.21.15.96:0/196077065 learned_addr learned my addr 172.21.15.96:0/196077065 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65f3fff640 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65f37fe640 1 -- 172.21.15.96:0/196077065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 msgr2=0x5560bc2d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65f37fe640 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65f37fe640 1 -- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 msgr2=0x5560bc2d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:09:42.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65f37fe640 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65f37fe640 1 -- 172.21.15.96:0/196077065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560bc2cb0f0 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65f37fe640 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d2790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f65e80c77d0 tx=0x7f65e803b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8310cefffccb27fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65d37fe640 1 -- 172.21.15.96:0/196077065 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8049070 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65d37fe640 1 -- 172.21.15.96:0/196077065 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f65e80ea0b0 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65ec016070 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65d37fe640 1 -- 172.21.15.96:0/196077065 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f65e80044b0 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f66017be740 10 client.15114 fetch_fsmap learned FSMap version 18 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5560bc2d6c40 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65d37fe640 1 -- 172.21.15.96:0/196077065 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ec016070 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.498+0000 7f65d37fe640 1 -- 172.21.15.96:0/196077065 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f65e80aece0 con 0x5560bbefc3b0 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f66017be740 10 client.15114 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f66017be740 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f66017be740 20 client.15114 populate_metadata read hostname 'smithi096' 2024-04-25T17:09:42.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f66017be740 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbf25270 0x5560bc349eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f66017be740 4 client.15114 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "status"}) -- 0x5560bc2331c0 con 0x5560bbf25270 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f65f3fff640 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbf25270 0x5560bc349eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f65d37fe640 10 client.15114 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f65f3fff640 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbf25270 0x5560bc349eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588380b3e0e0b16c server_cookie=d3d138314a3efc92 in_seq=0 out_seq=0 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f65d37fe640 1 -- 172.21.15.96:0/196077065 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5560bc2331c0 con 0x5560bbf25270 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f65d37fe640 10 client.15114 handle_command_reply: tid=0 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.499+0000 7f66017be740 1 client.15114 shutdown 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 msgr2=0x5560bc2d2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d2790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f65e80c77d0 tx=0x7f65e803b6b0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 shutdown_connections 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5560bbf25270 0x5560bc349eb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5560bbf04590 0x5560bc2d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65d37fe640 0 client.15114 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560bbefc3b0 0x5560bc2d2790 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 --2- 172.21.15.96:0/196077065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5560bbee3e40 0x5560bc2d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 >> 172.21.15.96:0/196077065 conn(0x5560bbf193c0 msgr2=0x5560bc2ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 shutdown_connections 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 1 -- 172.21.15.96:0/196077065 wait complete. 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f66017be740 20 client.15114 trim_cache size 0 max 16384 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "1be09d02-0325-11ef-bc93-c7b262605968", 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24521, 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T17:09:42.506 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T17:09:42.507 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 90.361773060999994, 2024-04-25T17:09:42.507 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 18, 2024-04-25T17:09:42.507 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T17:09:42.507 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T17:09:42.507 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 97.403034250999994 2024-04-25T17:09:42.507 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f65dc0afaf0 msgr2=0x7f65dc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f65dc0afaf0 0x7f65dc0b2000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5560bc2c7540 tx=0x7f65ec004a40 comp rx=0 tx=0).stop 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 msgr2=0x7f65f4199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4199950 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006010 tx=0x7f65e8004340 comp rx=0 tx=0).stop 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 shutdown_connections 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65f4108920 0x7f65f41a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f4103c20 0x7f65f4199950 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f65dc0afaf0 0x7f65dc0b2000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 --2- 172.21.15.96:0/794763002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f41029e0 0x7f65f4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 >> 172.21.15.96:0/794763002 conn(0x7f65f40fdbd0 msgr2=0x7f65f4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 shutdown_connections 2024-04-25T17:09:42.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:42.500+0000 7f65fa834640 1 -- 172.21.15.96:0/794763002 wait complete. 2024-04-25T17:09:42.514 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T17:09:42.514 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T17:09:42.522 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:09:42.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T17:09:42.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:42 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1453470525' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:09:42.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:42 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/181550767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:42.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:42 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/204575140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:09:42.757 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:43.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:43 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2172688443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:43.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:43 smithi096 ceph-mon[29131]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:43.469 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:43 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2172688443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:43.469 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:43 smithi097 ceph-mon[32602]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:43.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:43 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2172688443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:09:43.520 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:43 smithi171 ceph-mon[33189]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:44.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 -- 172.21.15.96:0/3354294649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c104ba0 msgr2=0x7fdf1c10b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 --2- 172.21.15.96:0/3354294649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c104ba0 0x7fdf1c10b670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400b0a0 tx=0x7fdf1400b3b0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 -- 172.21.15.96:0/3354294649 shutdown_connections 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 --2- 172.21.15.96:0/3354294649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c104ba0 0x7fdf1c10b670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 --2- 172.21.15.96:0/3354294649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf1c1036a0 0x7fdf1c103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 --2- 172.21.15.96:0/3354294649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf1c104080 0x7fdf1c104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 -- 172.21.15.96:0/3354294649 >> 172.21.15.96:0/3354294649 conn(0x7fdf1c0fded0 msgr2=0x7fdf1c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 -- 172.21.15.96:0/3354294649 shutdown_connections 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.573+0000 7fdf2212e700 1 -- 172.21.15.96:0/3354294649 wait complete. 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 Processor -- start 2024-04-25T17:09:44.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 -- start start 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c1036a0 0x7fdf1c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf1c104080 0x7fdf1c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf1c104ba0 0x7fdf1c1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c10eb80 con 0x7fdf1c1036a0 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c10ea00 con 0x7fdf1c104ba0 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.574+0000 7fdf2212e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c10ed00 con 0x7fdf1c104080 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c1036a0 0x7fdf1c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c1036a0 0x7fdf1c19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36698/0 (socket says 172.21.15.96:36698) 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 -- 172.21.15.96:0/4139182439 learned_addr learned my addr 172.21.15.96:0/4139182439 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1bfff700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf1c104ba0 0x7fdf1c1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:44.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1affd700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf1c104080 0x7fdf1c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 -- 172.21.15.96:0/4139182439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf1c104080 msgr2=0x7fdf1c19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf1c104080 0x7fdf1c19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 -- 172.21.15.96:0/4139182439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf1c104ba0 msgr2=0x7fdf1c1a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf1c104ba0 0x7fdf1c1a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 -- 172.21.15.96:0/4139182439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c1a22a0 con 0x7fdf1c1036a0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf1b7fe700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c1036a0 0x7fdf1c19a7d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00d950 tx=0x7fdf0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf18ff9700 1 -- 172.21.15.96:0/4139182439 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c017070 con 0x7fdf1c1036a0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf18ff9700 1 -- 172.21.15.96:0/4139182439 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0c009730 con 0x7fdf1c1036a0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1a2590 con 0x7fdf1c1036a0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf18ff9700 1 -- 172.21.15.96:0/4139182439 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c003a50 con 0x7fdf1c1036a0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.575+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a2ad0 con 0x7fdf1c1036a0 2024-04-25T17:09:44.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.576+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c051190 con 0x7fdf1c1036a0 2024-04-25T17:09:44.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.578+0000 7fdf18ff9700 1 -- 172.21.15.96:0/4139182439 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fdf0c003bf0 con 0x7fdf1c1036a0 2024-04-25T17:09:44.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.578+0000 7fdf18ff9700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf040afb80 0x7fdf040b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:44.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.579+0000 7fdf1affd700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf040afb80 0x7fdf040b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:44.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.579+0000 7fdf18ff9700 1 -- 172.21.15.96:0/4139182439 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fdf0c0deef0 con 0x7fdf1c1036a0 2024-04-25T17:09:44.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.579+0000 7fdf1affd700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf040afb80 0x7fdf040b2090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005ea0 tx=0x7fdf10005e30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:44.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.583+0000 7fdf18ff9700 1 -- 172.21.15.96:0/4139182439 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf0c020030 con 0x7fdf1c1036a0 2024-04-25T17:09:44.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.799+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fdf1c1a2df0 con 0x7fdf040afb80 2024-04-25T17:09:44.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.800+0000 7fdf18ff9700 1 -- 172.21.15.96:0/4139182439 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fdf1c1a2df0 con 0x7fdf040afb80 2024-04-25T17:09:44.800 INFO:teuthology.orchestra.run.smithi096.stdout:Backend: cephadm 2024-04-25T17:09:44.801 INFO:teuthology.orchestra.run.smithi096.stdout:Available: Yes 2024-04-25T17:09:44.801 INFO:teuthology.orchestra.run.smithi096.stdout:Paused: No 2024-04-25T17:09:44.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.802+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf040afb80 msgr2=0x7fdf040b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:44.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.802+0000 7fdf2212e700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf040afb80 0x7fdf040b2090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005ea0 tx=0x7fdf10005e30 comp rx=0 tx=0).stop 2024-04-25T17:09:44.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.802+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c1036a0 msgr2=0x7fdf1c19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:44.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.802+0000 7fdf2212e700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c1036a0 0x7fdf1c19a7d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00d950 tx=0x7fdf0c00dc60 comp rx=0 tx=0).stop 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 shutdown_connections 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf1c104ba0 0x7fdf1c1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fdf040afb80 0x7fdf040b2090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf1c1036a0 0x7fdf1c19a7d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 --2- 172.21.15.96:0/4139182439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdf1c104080 0x7fdf1c19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 >> 172.21.15.96:0/4139182439 conn(0x7fdf1c0fded0 msgr2=0x7fdf1c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 shutdown_connections 2024-04-25T17:09:44.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:44.803+0000 7fdf2212e700 1 -- 172.21.15.96:0/4139182439 wait complete. 2024-04-25T17:09:45.561 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T17:09:45.801 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:46.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:46 smithi096 ceph-mon[29131]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:46.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:46 smithi096 ceph-mon[29131]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:46 smithi097 ceph-mon[32602]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:46.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:46 smithi097 ceph-mon[32602]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:46.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:46 smithi171 ceph-mon[33189]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:46.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:46 smithi171 ceph-mon[33189]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:09:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 -- 172.21.15.96:0/2481249400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a41022b0 msgr2=0x7ff0a41026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/2481249400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a41022b0 0x7ff0a41026c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009990 tx=0x7ff08c009ca0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 -- 172.21.15.96:0/2481249400 shutdown_connections 2024-04-25T17:09:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/2481249400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a4102c00 0x7ff0a410c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/2481249400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a4105860 0x7ff0a4105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/2481249400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a41022b0 0x7ff0a41026c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 -- 172.21.15.96:0/2481249400 >> 172.21.15.96:0/2481249400 conn(0x7ff0a40fdf60 msgr2=0x7ff0a4100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 -- 172.21.15.96:0/2481249400 shutdown_connections 2024-04-25T17:09:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.465+0000 7ff0a9d79700 1 -- 172.21.15.96:0/2481249400 wait complete. 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 Processor -- start 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 -- start start 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a41022b0 0x7ff0a419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a4102c00 0x7ff0a419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a4105860 0x7ff0a41a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a410fd30 con 0x7ff0a4102c00 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a410fbb0 con 0x7ff0a41022b0 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.466+0000 7ff0a9d79700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a410feb0 con 0x7ff0a4105860 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a4102c00 0x7ff0a419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a4102c00 0x7ff0a419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37866/0 (socket says 172.21.15.96:37866) 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 -- 172.21.15.96:0/4234835634 learned_addr learned my addr 172.21.15.96:0/4234835634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff0a37fe700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a41022b0 0x7ff0a419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff0a3fff700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a4105860 0x7ff0a41a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 -- 172.21.15.96:0/4234835634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a4105860 msgr2=0x7ff0a41a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a4105860 0x7ff0a41a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 -- 172.21.15.96:0/4234835634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a41022b0 msgr2=0x7ff0a419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a41022b0 0x7ff0a419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 -- 172.21.15.96:0/4234835634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a41a2320 con 0x7ff0a4102c00 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff09affd700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a4102c00 0x7ff0a419ad90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff08c006010 tx=0x7ff08c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff0a17fa700 1 -- 172.21.15.96:0/4234835634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c027070 con 0x7ff0a4102c00 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a41a25b0 con 0x7ff0a4102c00 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff0a17fa700 1 -- 172.21.15.96:0/4234835634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08c0040c0 con 0x7ff0a4102c00 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff0a17fa700 1 -- 172.21.15.96:0/4234835634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c022d90 con 0x7ff0a4102c00 2024-04-25T17:09:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.467+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a41a2a60 con 0x7ff0a4102c00 2024-04-25T17:09:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.468+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a4051190 con 0x7ff0a4102c00 2024-04-25T17:09:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.470+0000 7ff0a17fa700 1 -- 172.21.15.96:0/4234835634 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff08c02a070 con 0x7ff0a4102c00 2024-04-25T17:09:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.470+0000 7ff0a17fa700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0900afb80 0x7ff0900b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:47.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.470+0000 7ff0a37fe700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0900afb80 0x7ff0900b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:47.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.471+0000 7ff0a17fa700 1 -- 172.21.15.96:0/4234835634 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7ff08c0dfce0 con 0x7ff0a4102c00 2024-04-25T17:09:47.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.471+0000 7ff0a37fe700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0900afb80 0x7ff0900b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0a406bed0 tx=0x7ff09400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:47.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.474+0000 7ff0a17fa700 1 -- 172.21.15.96:0/4234835634 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff08c013070 con 0x7ff0a4102c00 2024-04-25T17:09:47.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.680+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff0a4063ad0 con 0x7ff0900afb80 2024-04-25T17:09:47.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.694+0000 7ff0a17fa700 1 -- 172.21.15.96:0/4234835634 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7ff0a4063ad0 con 0x7ff0900afb80 2024-04-25T17:09:47.694 INFO:teuthology.orchestra.run.smithi096.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T17:09:47.694 INFO:teuthology.orchestra.run.smithi096.stdout:mds.a smithi096 running (105s) 16s ago 105s 81.2M - 19.0.0-3291-g994e2f20 d53c96779a29 6e4ee577a3d3 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.b smithi097 running (103s) 36s ago 102s 93.2M - 19.0.0-3291-g994e2f20 d53c96779a29 df6fe178286b 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.c smithi171 running (117s) 35s ago 116s 83.5M - 19.0.0-3291-g994e2f20 d53c96779a29 7e8a63c0c96f 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.d smithi096 running (2m) 16s ago 2m 81.6M - 19.0.0-3291-g994e2f20 d53c96779a29 eea5ecc06d43 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.e smithi097 running (2m) 36s ago 2m 80.7M - 19.0.0-3291-g994e2f20 d53c96779a29 5f451bd1b1d4 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.f smithi171 running (2m) 35s ago 119s 83.0M - 19.0.0-3291-g994e2f20 d53c96779a29 7d52ab227ea5 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.g smithi096 running (108s) 16s ago 108s 83.8M - 19.0.0-3291-g994e2f20 d53c96779a29 4ab184210c24 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.h smithi097 running (114s) 36s ago 113s 85.5M - 19.0.0-3291-g994e2f20 d53c96779a29 8ee9a110c540 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.i smithi171 running (111s) 35s ago 111s 82.8M - 19.0.0-3291-g994e2f20 d53c96779a29 4372dd765e2b 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.j smithi096 running (2m) 16s ago 2m 81.6M - 19.0.0-3291-g994e2f20 d53c96779a29 f8b3ae12fee4 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.k smithi097 running (2m) 36s ago 2m 83.8M - 19.0.0-3291-g994e2f20 d53c96779a29 900d491e8a39 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mds.l smithi171 running (2m) 35s ago 2m 83.9M - 19.0.0-3291-g994e2f20 d53c96779a29 fac5ba1796bc 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.x smithi096 *:9283,8765 running (9m) 16s ago 9m 583M - 19.0.0-3291-g994e2f20 d53c96779a29 a92cedda03df 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.y smithi097 *:8443,8765 running (7m) 36s ago 7m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 fb87bae10869 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.z smithi171 *:8443,8765 running (7m) 35s ago 7m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 8202b0e3e797 2024-04-25T17:09:47.695 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a smithi096 running (9m) 16s ago 9m 116M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 08b033812029 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b smithi097 running (7m) 36s ago 7m 106M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 1ea0d8872f77 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c smithi171 running (8m) 35s ago 8m 107M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 005b74a688e8 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 smithi096 running (7m) 16s ago 7m 111M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 ee49bd16414a 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 smithi097 running (6m) 36s ago 6m 123M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 5c82547aa2cd 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 smithi171 running (6m) 35s ago 6m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 fe7254a019dd 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 smithi096 running (6m) 16s ago 6m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 46ddbe251f8a 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 smithi097 running (5m) 36s ago 5m 119M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 55bd02cebc10 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 smithi171 running (5m) 35s ago 5m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 bfd33f2472ea 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 smithi096 running (4m) 16s ago 4m 108M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 f131721884f1 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 smithi097 running (4m) 36s ago 4m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 a5f7d62f56a0 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 smithi171 running (3m) 35s ago 3m 120M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 25d0e7bbd679 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 smithi096 running (3m) 16s ago 3m 118M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 aae4a54d3bbe 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 smithi097 running (3m) 36s ago 2m 113M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 7ae6eb61db11 2024-04-25T17:09:47.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 smithi171 running (2m) 35s ago 2m 107M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 f5cd755b13b6 2024-04-25T17:09:47.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.696+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0900afb80 msgr2=0x7ff0900b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:47.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.696+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0900afb80 0x7ff0900b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0a406bed0 tx=0x7ff09400b410 comp rx=0 tx=0).stop 2024-04-25T17:09:47.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.696+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a4102c00 msgr2=0x7ff0a419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:47.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.696+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a4102c00 0x7ff0a419ad90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff08c006010 tx=0x7ff08c0047b0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 shutdown_connections 2024-04-25T17:09:47.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0a4105860 0x7ff0a41a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a41022b0 0x7ff0a419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7ff0900afb80 0x7ff0900b2090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 --2- 172.21.15.96:0/4234835634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0a4102c00 0x7ff0a419ad90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:47.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 >> 172.21.15.96:0/4234835634 conn(0x7ff0a40fdf60 msgr2=0x7ff0a4108800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:47.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 shutdown_connections 2024-04-25T17:09:47.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:47.697+0000 7ff0a9d79700 1 -- 172.21.15.96:0/4234835634 wait complete. 2024-04-25T17:09:48.376 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T17:09:48.398 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:48 smithi096 ceph-mon[29131]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T17:09:48.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:48 smithi097 ceph-mon[32602]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T17:09:48.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:48 smithi171 ceph-mon[33189]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T17:09:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:49.406 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:49 smithi096 ceph-mon[29131]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:49.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:49 smithi097 ceph-mon[32602]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:49.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:49 smithi171 ceph-mon[33189]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 -- 172.21.15.96:0/1660468942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844104bf0 msgr2=0x7fe84410b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 --2- 172.21.15.96:0/1660468942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844104bf0 0x7fe84410b4a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00b0a0 tx=0x7fe83c00b3b0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 -- 172.21.15.96:0/1660468942 shutdown_connections 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 --2- 172.21.15.96:0/1660468942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844104bf0 0x7fe84410b4a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 --2- 172.21.15.96:0/1660468942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe844103120 0x7fe844103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 --2- 172.21.15.96:0/1660468942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe844104250 0x7fe844104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 -- 172.21.15.96:0/1660468942 >> 172.21.15.96:0/1660468942 conn(0x7fe8440fe760 msgr2=0x7fe844100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 -- 172.21.15.96:0/1660468942 shutdown_connections 2024-04-25T17:09:50.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.272+0000 7fe8491ae700 1 -- 172.21.15.96:0/1660468942 wait complete. 2024-04-25T17:09:50.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.273+0000 7fe8491ae700 1 Processor -- start 2024-04-25T17:09:50.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.273+0000 7fe8491ae700 1 -- start start 2024-04-25T17:09:50.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe8491ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844103120 0x7fe844076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:50.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe8491ae700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe844104250 0x7fe84419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:50.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe8491ae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe844104bf0 0x7fe8441a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:50.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe8491ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe84410ec10 con 0x7fe844103120 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe8491ae700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe84410ea90 con 0x7fe844104bf0 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe8491ae700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe84410ed90 con 0x7fe844104250 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844103120 0x7fe844076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844103120 0x7fe844076320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37872/0 (socket says 172.21.15.96:37872) 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 -- 172.21.15.96:0/3490774809 learned_addr learned my addr 172.21.15.96:0/3490774809 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe84259c700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe844104250 0x7fe84419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe84359e700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe844104bf0 0x7fe8441a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 -- 172.21.15.96:0/3490774809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe844104250 msgr2=0x7fe84419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe844104250 0x7fe84419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 -- 172.21.15.96:0/3490774809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe844104bf0 msgr2=0x7fe8441a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe844104bf0 0x7fe8441a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 -- 172.21.15.96:0/3490774809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8441a2350 con 0x7fe844103120 2024-04-25T17:09:50.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe842d9d700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844103120 0x7fe844076320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe83400d950 tx=0x7fe83400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:50.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.274+0000 7fe82bfff700 1 -- 172.21.15.96:0/3490774809 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834017070 con 0x7fe844103120 2024-04-25T17:09:50.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.275+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8441a2640 con 0x7fe844103120 2024-04-25T17:09:50.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.275+0000 7fe82bfff700 1 -- 172.21.15.96:0/3490774809 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe83400de30 con 0x7fe844103120 2024-04-25T17:09:50.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.275+0000 7fe82bfff700 1 -- 172.21.15.96:0/3490774809 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834003b10 con 0x7fe844103120 2024-04-25T17:09:50.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.275+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8441a2c50 con 0x7fe844103120 2024-04-25T17:09:50.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.275+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe844051190 con 0x7fe844103120 2024-04-25T17:09:50.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.277+0000 7fe82bfff700 1 -- 172.21.15.96:0/3490774809 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe834003cb0 con 0x7fe844103120 2024-04-25T17:09:50.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.277+0000 7fe82bfff700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe82c0afb80 0x7fe82c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:50.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.278+0000 7fe84259c700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe82c0afb80 0x7fe82c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:50.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.278+0000 7fe82bfff700 1 -- 172.21.15.96:0/3490774809 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fe8340def60 con 0x7fe844103120 2024-04-25T17:09:50.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.278+0000 7fe84259c700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe82c0afb80 0x7fe82c0b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe838006fd0 tx=0x7fe838008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:50.278 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:50 smithi096 ceph-mon[29131]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T17:09:50.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.281+0000 7fe82bfff700 1 -- 172.21.15.96:0/3490774809 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe834020030 con 0x7fe844103120 2024-04-25T17:09:50.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:50 smithi097 ceph-mon[32602]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T17:09:50.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.488+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe844063ad0 con 0x7fe82c0afb80 2024-04-25T17:09:50.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.494+0000 7fe82bfff700 1 -- 172.21.15.96:0/3490774809 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe844063ad0 con 0x7fe82c0afb80 2024-04-25T17:09:50.494 INFO:teuthology.orchestra.run.smithi096.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T17:09:50.494 INFO:teuthology.orchestra.run.smithi096.stdout:mds.all 12/12 38s ago 2m smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-25T17:09:50.494 INFO:teuthology.orchestra.run.smithi096.stdout:mgr 3/3 38s ago 7m smithi096=x;smithi097=y;smithi171=z;count:3 2024-04-25T17:09:50.494 INFO:teuthology.orchestra.run.smithi096.stdout:mon 3/3 38s ago 8m smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi171:172.21.15.171=c;count:3 2024-04-25T17:09:50.494 INFO:teuthology.orchestra.run.smithi096.stdout:osd 12 38s ago - 2024-04-25T17:09:50.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe82c0afb80 msgr2=0x7fe82c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:50.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe82c0afb80 0x7fe82c0b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe838006fd0 tx=0x7fe838008040 comp rx=0 tx=0).stop 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844103120 msgr2=0x7fe844076320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844103120 0x7fe844076320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe83400d950 tx=0x7fe83400dd10 comp rx=0 tx=0).stop 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 shutdown_connections 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe844104bf0 0x7fe8441a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe82c0afb80 0x7fe82c0b2090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe844103120 0x7fe844076320 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 --2- 172.21.15.96:0/3490774809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe844104250 0x7fe84419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 >> 172.21.15.96:0/3490774809 conn(0x7fe8440fe760 msgr2=0x7fe844107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.496+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 shutdown_connections 2024-04-25T17:09:50.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:50.497+0000 7fe8491ae700 1 -- 172.21.15.96:0/3490774809 wait complete. 2024-04-25T17:09:50.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:50 smithi171 ceph-mon[33189]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T17:09:51.193 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T17:09:51.471 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:52.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:52 smithi096 ceph-mon[29131]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:52.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:52 smithi096 ceph-mon[29131]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:52 smithi097 ceph-mon[32602]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:52.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:52 smithi097 ceph-mon[32602]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:52.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:52 smithi171 ceph-mon[33189]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:52.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:52 smithi171 ceph-mon[33189]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 -- 172.21.15.96:0/2098758095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104210 msgr2=0x7facb8104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 --2- 172.21.15.96:0/2098758095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104210 0x7facb8104620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7facac009990 tx=0x7facac009ca0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 -- 172.21.15.96:0/2098758095 shutdown_connections 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 --2- 172.21.15.96:0/2098758095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8104bb0 0x7facb810b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 --2- 172.21.15.96:0/2098758095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facb81030e0 0x7facb81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 --2- 172.21.15.96:0/2098758095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104210 0x7facb8104620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 -- 172.21.15.96:0/2098758095 >> 172.21.15.96:0/2098758095 conn(0x7facb80fe740 msgr2=0x7facb8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 -- 172.21.15.96:0/2098758095 shutdown_connections 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 -- 172.21.15.96:0/2098758095 wait complete. 2024-04-25T17:09:53.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 Processor -- start 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.268+0000 7facbde67700 1 -- start start 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facbde67700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facb81030e0 0x7facb819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facbde67700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8104210 0x7facb819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facbde67700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104bb0 0x7facb81a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facbde67700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7facb810ebd0 con 0x7facb8104bb0 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facbde67700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7facb810ea50 con 0x7facb8104210 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facbde67700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7facb810ed50 con 0x7facb81030e0 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104bb0 0x7facb81a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104bb0 0x7facb81a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37898/0 (socket says 172.21.15.96:37898) 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 -- 172.21.15.96:0/2251158192 learned_addr learned my addr 172.21.15.96:0/2251158192 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb77fe700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facb81030e0 0x7facb819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb6ffd700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8104210 0x7facb819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 -- 172.21.15.96:0/2251158192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facb81030e0 msgr2=0x7facb819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:53.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facb81030e0 0x7facb819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 -- 172.21.15.96:0/2251158192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8104210 msgr2=0x7facb819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8104210 0x7facb819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 -- 172.21.15.96:0/2251158192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb81a2360 con 0x7facb8104bb0 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb7fff700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104bb0 0x7facb81a1ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7facb000b970 tx=0x7facb000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.269+0000 7facb4ff9700 1 -- 172.21.15.96:0/2251158192 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0019070 con 0x7facb8104bb0 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.270+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb81a2650 con 0x7facb8104bb0 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.270+0000 7facb4ff9700 1 -- 172.21.15.96:0/2251158192 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb000cde0 con 0x7facb8104bb0 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.270+0000 7facb4ff9700 1 -- 172.21.15.96:0/2251158192 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0007370 con 0x7facb8104bb0 2024-04-25T17:09:53.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.270+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb81a2c10 con 0x7facb8104bb0 2024-04-25T17:09:53.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.270+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb8051190 con 0x7facb8104bb0 2024-04-25T17:09:53.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.272+0000 7facb4ff9700 1 -- 172.21.15.96:0/2251158192 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7facb001c070 con 0x7facb8104bb0 2024-04-25T17:09:53.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.272+0000 7facb4ff9700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7faca00afb80 0x7faca00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:53.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.273+0000 7facb77fe700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7faca00afb80 0x7faca00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:53.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.273+0000 7facb4ff9700 1 -- 172.21.15.96:0/2251158192 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7facb00de210 con 0x7facb8104bb0 2024-04-25T17:09:53.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.273+0000 7facb77fe700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7faca00afb80 0x7faca00b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7facb8104110 tx=0x7faca800a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:53.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.276+0000 7facb4ff9700 1 -- 172.21.15.96:0/2251158192 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facb00a6df0 con 0x7facb8104bb0 2024-04-25T17:09:53.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.484+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7facb81a2ed0 con 0x7faca00afb80 2024-04-25T17:09:53.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.485+0000 7facb4ff9700 1 -- 172.21.15.96:0/2251158192 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7facb81a2ed0 con 0x7faca00afb80 2024-04-25T17:09:53.485 INFO:teuthology.orchestra.run.smithi096.stdout:HOST ADDR LABELS STATUS 2024-04-25T17:09:53.485 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 172.21.15.96 2024-04-25T17:09:53.485 INFO:teuthology.orchestra.run.smithi096.stdout:smithi097 172.21.15.97 2024-04-25T17:09:53.486 INFO:teuthology.orchestra.run.smithi096.stdout:smithi171 172.21.15.171 2024-04-25T17:09:53.486 INFO:teuthology.orchestra.run.smithi096.stdout:3 hosts in cluster 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.487+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7faca00afb80 msgr2=0x7faca00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.487+0000 7facbde67700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7faca00afb80 0x7faca00b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7facb8104110 tx=0x7faca800a380 comp rx=0 tx=0).stop 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.487+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104bb0 msgr2=0x7facb81a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.487+0000 7facbde67700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104bb0 0x7facb81a1ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7facb000b970 tx=0x7facb000bc80 comp rx=0 tx=0).stop 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 shutdown_connections 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7faca00afb80 0x7faca00b2090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facb8104bb0 0x7facb81a1ce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facb81030e0 0x7facb819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 --2- 172.21.15.96:0/2251158192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8104210 0x7facb819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 >> 172.21.15.96:0/2251158192 conn(0x7facb80fe740 msgr2=0x7facb8100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 shutdown_connections 2024-04-25T17:09:53.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:53.488+0000 7facbde67700 1 -- 172.21.15.96:0/2251158192 wait complete. 2024-04-25T17:09:54.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T17:09:54.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:54 smithi096 ceph-mon[29131]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:54.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:54 smithi097 ceph-mon[32602]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:54.515 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:54.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:54 smithi171 ceph-mon[33189]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:55.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:55 smithi096 ceph-mon[29131]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:55.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:55 smithi097 ceph-mon[32602]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:55.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:55 smithi171 ceph-mon[33189]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:56.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:56 smithi096 ceph-mon[29131]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.426+0000 7f572a6a1700 1 -- 172.21.15.96:0/158211999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57241028b0 msgr2=0x7f5724102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.426+0000 7f572a6a1700 1 --2- 172.21.15.96:0/158211999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57241028b0 0x7f5724102cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f5714009ca0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 -- 172.21.15.96:0/158211999 shutdown_connections 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 --2- 172.21.15.96:0/158211999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5724104380 0x7f572410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 --2- 172.21.15.96:0/158211999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57241028b0 0x7f5724102cc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 --2- 172.21.15.96:0/158211999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241039e0 0x7f5724103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 -- 172.21.15.96:0/158211999 >> 172.21.15.96:0/158211999 conn(0x7f57240fdef0 msgr2=0x7f57241003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 -- 172.21.15.96:0/158211999 shutdown_connections 2024-04-25T17:09:56.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 -- 172.21.15.96:0/158211999 wait complete. 2024-04-25T17:09:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.427+0000 7f572a6a1700 1 Processor -- start 2024-04-25T17:09:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f572a6a1700 1 -- start start 2024-04-25T17:09:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f572a6a1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57241028b0 0x7f57241997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f572a6a1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241039e0 0x7f5724199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f572a6a1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104380 0x7f57241a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f572a6a1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f572410e370 con 0x7f5724104380 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f572a6a1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f572410e1f0 con 0x7f57241028b0 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f572a6a1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f572410e4f0 con 0x7f57241039e0 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104380 0x7f57241a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104380 0x7f57241a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37920/0 (socket says 172.21.15.96:37920) 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 -- 172.21.15.96:0/3702068447 learned_addr learned my addr 172.21.15.96:0/3702068447 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f57237fe700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241039e0 0x7f5724199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5723fff700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57241028b0 0x7f57241997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 -- 172.21.15.96:0/3702068447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241039e0 msgr2=0x7f5724199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241039e0 0x7f5724199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 -- 172.21.15.96:0/3702068447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57241028b0 msgr2=0x7f57241997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:56.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57241028b0 0x7f57241997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5728c3e700 1 -- 172.21.15.96:0/3702068447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57241a1330 con 0x7f5724104380 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f57237fe700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241039e0 0x7f5724199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.428+0000 7f5723fff700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57241028b0 0x7f57241997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.429+0000 7f5728c3e700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104380 0x7f57241a0cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f571c00ada0 tx=0x7f571c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.429+0000 7f57217fa700 1 -- 172.21.15.96:0/3702068447 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c017020 con 0x7f5724104380 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.429+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57241a1620 con 0x7f5724104380 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.429+0000 7f57217fa700 1 -- 172.21.15.96:0/3702068447 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f571c00ce30 con 0x7f5724104380 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.429+0000 7f57217fa700 1 -- 172.21.15.96:0/3702068447 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0054a0 con 0x7f5724104380 2024-04-25T17:09:56.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.429+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57241a1be0 con 0x7f5724104380 2024-04-25T17:09:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.430+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5724051190 con 0x7f5724104380 2024-04-25T17:09:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.431+0000 7f57217fa700 1 -- 172.21.15.96:0/3702068447 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f571c00c980 con 0x7f5724104380 2024-04-25T17:09:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.432+0000 7f57217fa700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f570c0afb80 0x7f570c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.432+0000 7f5723fff700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f570c0afb80 0x7f570c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.432+0000 7f57217fa700 1 -- 172.21.15.96:0/3702068447 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f571c0dea50 con 0x7f5724104380 2024-04-25T17:09:56.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.433+0000 7f5723fff700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f570c0afb80 0x7f570c0b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f571400b5c0 tx=0x7f5714005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:56.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.436+0000 7f57217fa700 1 -- 172.21.15.96:0/3702068447 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f571c021020 con 0x7f5724104380 2024-04-25T17:09:56.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:56 smithi097 ceph-mon[32602]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:56.529 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:56 smithi171 ceph-mon[33189]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:09:56.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.644+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f5724063ad0 con 0x7f570c0afb80 2024-04-25T17:09:56.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.645+0000 7f57217fa700 1 -- 172.21.15.96:0/3702068447 <== mgr.14148 v2:172.21.15.96:6800/3530143863 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f5724063ad0 con 0x7f570c0afb80 2024-04-25T17:09:56.645 INFO:teuthology.orchestra.run.smithi096.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T17:09:56.645 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T17:09:56.646 INFO:teuthology.orchestra.run.smithi096.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T17:09:56.646 INFO:teuthology.orchestra.run.smithi096.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T17:09:56.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.647+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f570c0afb80 msgr2=0x7f570c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.647+0000 7f572a6a1700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f570c0afb80 0x7f570c0b2090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f571400b5c0 tx=0x7f5714005a90 comp rx=0 tx=0).stop 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.647+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104380 msgr2=0x7f57241a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.647+0000 7f572a6a1700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104380 0x7f57241a0cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f571c00ada0 tx=0x7f571c00c330 comp rx=0 tx=0).stop 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 shutdown_connections 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f570c0afb80 0x7f570c0b2090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104380 0x7f57241a0cb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57241028b0 0x7f57241997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 --2- 172.21.15.96:0/3702068447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241039e0 0x7f5724199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 >> 172.21.15.96:0/3702068447 conn(0x7f57240fdef0 msgr2=0x7f5724107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 shutdown_connections 2024-04-25T17:09:56.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:56.648+0000 7f572a6a1700 1 -- 172.21.15.96:0/3702068447 wait complete. 2024-04-25T17:09:57.397 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T17:09:57.407 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:09:57.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T17:09:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:09:58.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:58 smithi096 ceph-mon[29131]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:58.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:58 smithi096 ceph-mon[29131]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:58 smithi097 ceph-mon[32602]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:58.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:58 smithi097 ceph-mon[32602]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:58.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:58 smithi171 ceph-mon[33189]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:09:58.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:58 smithi171 ceph-mon[33189]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 -- 172.21.15.96:0/1355927295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 msgr2=0x7faccc10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 --2- 172.21.15.96:0/1355927295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc10b460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7facc400b0a0 tx=0x7facc400b3b0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 -- 172.21.15.96:0/1355927295 shutdown_connections 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 --2- 172.21.15.96:0/1355927295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc10b460 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 --2- 172.21.15.96:0/1355927295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faccc1030e0 0x7faccc1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 --2- 172.21.15.96:0/1355927295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faccc104210 0x7faccc104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 -- 172.21.15.96:0/1355927295 >> 172.21.15.96:0/1355927295 conn(0x7faccc0fe740 msgr2=0x7faccc100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 -- 172.21.15.96:0/1355927295 shutdown_connections 2024-04-25T17:09:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 -- 172.21.15.96:0/1355927295 wait complete. 2024-04-25T17:09:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 Processor -- start 2024-04-25T17:09:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.274+0000 7facd4071700 1 -- start start 2024-04-25T17:09:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faccc1030e0 0x7faccc199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faccc104210 0x7faccc199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faccc10eb60 con 0x7faccc104bb0 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faccc10e9e0 con 0x7faccc1030e0 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faccc10ece0 con 0x7faccc104210 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57892/0 (socket says 172.21.15.96:57892) 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 -- 172.21.15.96:0/740235261 learned_addr learned my addr 172.21.15.96:0/740235261 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd1e0d700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faccc1030e0 0x7faccc199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd160c700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faccc104210 0x7faccc199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 -- 172.21.15.96:0/740235261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faccc104210 msgr2=0x7faccc199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faccc104210 0x7faccc199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 -- 172.21.15.96:0/740235261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faccc1030e0 msgr2=0x7faccc199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faccc1030e0 0x7faccc199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 -- 172.21.15.96:0/740235261 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc1a11b0 con 0x7faccc104bb0 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd1e0d700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faccc1030e0 0x7faccc199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:09:59.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd260e700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc1a0b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7facc40062a0 tx=0x7facc4004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:59.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facbeffd700 1 -- 172.21.15.96:0/740235261 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4021070 con 0x7faccc104bb0 2024-04-25T17:09:59.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc0758c0 con 0x7faccc104bb0 2024-04-25T17:09:59.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facbeffd700 1 -- 172.21.15.96:0/740235261 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc4004020 con 0x7faccc104bb0 2024-04-25T17:09:59.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facbeffd700 1 -- 172.21.15.96:0/740235261 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc401d670 con 0x7faccc104bb0 2024-04-25T17:09:59.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.275+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc075e50 con 0x7faccc104bb0 2024-04-25T17:09:59.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.276+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faccc051190 con 0x7faccc104bb0 2024-04-25T17:09:59.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.278+0000 7facbeffd700 1 -- 172.21.15.96:0/740235261 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7facc40044d0 con 0x7faccc104bb0 2024-04-25T17:09:59.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.278+0000 7facbeffd700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7facb80afb80 0x7facb80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:09:59.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.278+0000 7facd1e0d700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7facb80afb80 0x7facb80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:09:59.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.279+0000 7facbeffd700 1 -- 172.21.15.96:0/740235261 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7facc400f020 con 0x7faccc104bb0 2024-04-25T17:09:59.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.279+0000 7facd1e0d700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7facb80afb80 0x7facb80b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facc0005950 tx=0x7facc00058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:09:59.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.282+0000 7facbeffd700 1 -- 172.21.15.96:0/740235261 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facc402a020 con 0x7faccc104bb0 2024-04-25T17:09:59.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.529+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7faccc109ce0 con 0x7faccc104bb0 2024-04-25T17:09:59.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.529+0000 7facbeffd700 1 -- 172.21.15.96:0/740235261 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) ==== 76+0+3006 (secure 0 0 0) 0x7facc40af070 con 0x7faccc104bb0 2024-04-25T17:09:59.529 INFO:teuthology.orchestra.run.smithi096.stdout:e18 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:legacy client fscid: 1 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:Filesystem 'cephfs' (1) 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:fs_name cephfs 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 18 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-25T17:08:11.494613+0000 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-04-25T17:08:14.735284+0000 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:tableserver 0 2024-04-25T17:09:59.530 INFO:teuthology.orchestra.run.smithi096.stdout:root 0 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:session_timeout 300 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:session_autoclose 300 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:max_file_size 1099511627776 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:max_xattr_size 65536 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:required_client_features {} 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure 0 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure_osd_epoch 0 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:max_mds 1 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:in 0 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:up {0=24521} 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:failed 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:damaged 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:stopped 2024-04-25T17:09:59.531 INFO:teuthology.orchestra.run.smithi096.stdout:data_pools [3] 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:metadata_pool 2 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:inline_data disabled 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:balancer 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:bal_rank_mask -1 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:standby_count_wanted 0 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:qdb_cluster leader: 24521 members: 24521 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.h{0:24485} state up:standby-replay seq 1 addr [v2:172.21.15.97:6836/825277463,v1:172.21.15.97:6837/825277463] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:Standby daemons: 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.j{-1:14571} state up:standby seq 1 addr [v2:172.21.15.96:6834/2619590214,v1:172.21.15.96:6835/2619590214] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.d{-1:14583} state up:standby seq 1 addr [v2:172.21.15.96:6836/2018971567,v1:172.21.15.96:6837/2018971567] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.532 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.g{-1:14655} state up:standby seq 1 addr [v2:172.21.15.96:6838/416971885,v1:172.21.15.96:6839/416971885] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.533 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a{-1:14667} state up:standby seq 1 addr [v2:172.21.15.96:6840/1046285931,v1:172.21.15.96:6841/1046285931] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.533 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.l{-1:24403} state up:standby seq 1 addr [v2:172.21.15.171:6832/3848941544,v1:172.21.15.171:6833/3848941544] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.533 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.k{-1:24425} state up:standby seq 1 addr [v2:172.21.15.97:6832/3336646389,v1:172.21.15.97:6833/3336646389] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.533 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.f{-1:24445} state up:standby seq 1 addr [v2:172.21.15.171:6834/1302443885,v1:172.21.15.171:6835/1302443885] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.533 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c{-1:24457} state up:standby seq 1 addr [v2:172.21.15.171:6836/1065540086,v1:172.21.15.171:6837/1065540086] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.533 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.e{-1:24461} state up:standby seq 1 addr [v2:172.21.15.97:6834/3531324294,v1:172.21.15.97:6835/3531324294] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.533 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.i{-1:24478} state up:standby seq 1 addr [v2:172.21.15.171:6838/2758729388,v1:172.21.15.171:6839/2758729388] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.531+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7facb80afb80 msgr2=0x7facb80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.531+0000 7facd4071700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7facb80afb80 0x7facb80b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facc0005950 tx=0x7facc00058e0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 msgr2=0x7faccc1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc1a0b30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7facc40062a0 tx=0x7facc4004ef0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 shutdown_connections 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7facb80afb80 0x7facb80b2090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faccc104bb0 0x7faccc1a0b30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faccc1030e0 0x7faccc199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 --2- 172.21.15.96:0/740235261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faccc104210 0x7faccc199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:09:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 >> 172.21.15.96:0/740235261 conn(0x7faccc0fe740 msgr2=0x7faccc107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:09:59.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 shutdown_connections 2024-04-25T17:09:59.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:09:59.532+0000 7facd4071700 1 -- 172.21.15.96:0/740235261 wait complete. 2024-04-25T17:09:59.535 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-04-25T17:09:59.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:09:59 smithi096 ceph-mon[29131]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:59.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:09:59 smithi097 ceph-mon[32602]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:09:59.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:09:59 smithi171 ceph-mon[33189]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:00.300 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T17:10:00.551 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:10:00.561 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:00 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/740235261' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T17:10:00.561 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:00 smithi096 ceph-mon[29131]: overall HEALTH_OK 2024-04-25T17:10:00.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:00 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/740235261' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T17:10:00.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:00 smithi097 ceph-mon[32602]: overall HEALTH_OK 2024-04-25T17:10:00.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:00 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/740235261' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T17:10:00.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:00 smithi171 ceph-mon[33189]: overall HEALTH_OK 2024-04-25T17:10:01.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:01 smithi096 ceph-mon[29131]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:10:01.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:01 smithi097 ceph-mon[32602]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:10:01.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:01 smithi171 ceph-mon[33189]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:10:02.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.274+0000 7f0c279d7700 1 -- 172.21.15.96:0/4030397260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104bb0 msgr2=0x7f0c2010b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.274+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4030397260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104bb0 0x7f0c2010b460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800b0a0 tx=0x7f0c1800b3b0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 -- 172.21.15.96:0/4030397260 shutdown_connections 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4030397260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104bb0 0x7f0c2010b460 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4030397260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c201030e0 0x7f0c201034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4030397260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c20104210 0x7f0c20104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 -- 172.21.15.96:0/4030397260 >> 172.21.15.96:0/4030397260 conn(0x7f0c200fe740 msgr2=0x7f0c20100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 -- 172.21.15.96:0/4030397260 shutdown_connections 2024-04-25T17:10:02.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 -- 172.21.15.96:0/4030397260 wait complete. 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 Processor -- start 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.275+0000 7f0c279d7700 1 -- start start 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c279d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c201030e0 0x7f0c20076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c279d7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104210 0x7f0c2019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c279d7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c20104bb0 0x7f0c201a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c279d7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2010ec90 con 0x7f0c20104210 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c279d7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2010eb10 con 0x7f0c201030e0 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c279d7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2010ee10 con 0x7f0c20104bb0 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104210 0x7f0c2019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104210 0x7f0c2019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57910/0 (socket says 172.21.15.96:57910) 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 -- 172.21.15.96:0/4041992168 learned_addr learned my addr 172.21.15.96:0/4041992168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c25f74700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c20104bb0 0x7f0c201a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c25773700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c201030e0 0x7f0c20076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:02.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 -- 172.21.15.96:0/4041992168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c20104bb0 msgr2=0x7f0c201a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c20104bb0 0x7f0c201a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 -- 172.21.15.96:0/4041992168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c201030e0 msgr2=0x7f0c20076320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c201030e0 0x7f0c20076320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 -- 172.21.15.96:0/4041992168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c201a2440 con 0x7f0c20104210 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c24f72700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104210 0x7f0c2019f6a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000b530 tx=0x7f0c1000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.276+0000 7f0c167fc700 1 -- 172.21.15.96:0/4041992168 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10015020 con 0x7f0c20104210 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.277+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c201a2730 con 0x7f0c20104210 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.277+0000 7f0c167fc700 1 -- 172.21.15.96:0/4041992168 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c10011ce0 con 0x7f0c20104210 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.277+0000 7f0c167fc700 1 -- 172.21.15.96:0/4041992168 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10010dd0 con 0x7f0c20104210 2024-04-25T17:10:02.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.277+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c201a2d40 con 0x7f0c20104210 2024-04-25T17:10:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.277+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c20051190 con 0x7f0c20104210 2024-04-25T17:10:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.279+0000 7f0c167fc700 1 -- 172.21.15.96:0/4041992168 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0c10018020 con 0x7f0c20104210 2024-04-25T17:10:02.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.280+0000 7f0c167fc700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0c0c0afb80 0x7f0c0c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.280+0000 7f0c25773700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0c0c0afb80 0x7f0c0c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.280+0000 7f0c167fc700 1 -- 172.21.15.96:0/4041992168 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f0c100de880 con 0x7f0c20104210 2024-04-25T17:10:02.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.280+0000 7f0c25773700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0c0c0afb80 0x7f0c0c0b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0c20104110 tx=0x7f0c1c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:02.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.283+0000 7f0c167fc700 1 -- 172.21.15.96:0/4041992168 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c1001f020 con 0x7f0c20104210 2024-04-25T17:10:02.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.487+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f0c2002e510 con 0x7f0c20104210 2024-04-25T17:10:02.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.488+0000 7f0c167fc700 1 -- 172.21.15.96:0/4041992168 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v78) ==== 56+0+5292 (secure 0 0 0) 0x7f0c2002e510 con 0x7f0c20104210 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 78 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:fsid 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-25T17:00:08.429242+0000 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-04-25T17:09:04.141423+0000 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:crush_version 26 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:full_ratio 0.95 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:backfillfull_ratio 0.9 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:nearfull_ratio 0.85 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:require_min_compat_client luminous 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:min_compat_client luminous 2024-04-25T17:10:02.488 INFO:teuthology.orchestra.run.smithi096.stdout:require_osd_release squid 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:stretch_mode_enabled false 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:max_osd 12 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 up in weight 1 up_from 8 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6802/1323373413,v1:172.21.15.96:6803/1323373413] [v2:172.21.15.96:6804/1323373413,v1:172.21.15.96:6805/1323373413] exists,up b0086d3f-f5b8-46e8-95db-03996e609f38 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 up in weight 1 up_from 13 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/1211116017,v1:172.21.15.97:6801/1211116017] [v2:172.21.15.97:6802/1211116017,v1:172.21.15.97:6803/1211116017] exists,up 7e42e40f-3157-4e43-b9fc-689bacb722b3 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 up in weight 1 up_from 18 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/3917754473,v1:172.21.15.171:6801/3917754473] [v2:172.21.15.171:6802/3917754473,v1:172.21.15.171:6803/3917754473] exists,up 4a1d306b-692d-446a-a5af-40c55c06522e 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 up in weight 1 up_from 25 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6810/3604096933,v1:172.21.15.96:6811/3604096933] [v2:172.21.15.96:6812/3604096933,v1:172.21.15.96:6813/3604096933] exists,up abdaf538-404e-4714-b1a2-dcc2c22f275b 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 up in weight 1 up_from 31 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/2529478189,v1:172.21.15.97:6809/2529478189] [v2:172.21.15.97:6810/2529478189,v1:172.21.15.97:6811/2529478189] exists,up 27140752-dc84-4eb3-a875-5637f11a800a 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 up in weight 1 up_from 36 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/651283104,v1:172.21.15.171:6809/651283104] [v2:172.21.15.171:6810/651283104,v1:172.21.15.171:6811/651283104] exists,up d623b6a2-12d2-4564-a836-e726059745f9 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 up in weight 1 up_from 42 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6818/2945403945,v1:172.21.15.96:6819/2945403945] [v2:172.21.15.96:6820/2945403945,v1:172.21.15.96:6821/2945403945] exists,up 3f6a0ae2-7293-440b-9fcc-385526e69434 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 up in weight 1 up_from 48 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2873132021,v1:172.21.15.97:6817/2873132021] [v2:172.21.15.97:6818/2873132021,v1:172.21.15.97:6819/2873132021] exists,up 2281b753-0cf4-41de-b12c-9dda1afa5c71 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 up in weight 1 up_from 54 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/2677669720,v1:172.21.15.171:6817/2677669720] [v2:172.21.15.171:6818/2677669720,v1:172.21.15.171:6819/2677669720] exists,up 521eeba2-7a5c-44a5-839e-d57d391c2a75 2024-04-25T17:10:02.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 up in weight 1 up_from 60 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6826/454601194,v1:172.21.15.96:6827/454601194] [v2:172.21.15.96:6828/454601194,v1:172.21.15.96:6829/454601194] exists,up 9d96f5d9-4be8-445e-a878-b9c296673d1f 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 up in weight 1 up_from 65 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/213999228,v1:172.21.15.97:6825/213999228] [v2:172.21.15.97:6826/213999228,v1:172.21.15.97:6827/213999228] exists,up 50d42cae-d4f6-4e59-b0c0-8678555c6b4b 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 up in weight 1 up_from 71 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/2756841821,v1:172.21.15.171:6825/2756841821] [v2:172.21.15.171:6826/2756841821,v1:172.21.15.171:6827/2756841821] exists,up fec892a1-61fb-43aa-96db-30f1a3e58416 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.b [2,4] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.1a [8,4] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.1b [5,11] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.1d [2,6] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.1f [8,1] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.e [5,9] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.36 [4,9] 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/4231768717 expires 2024-04-26T17:01:02.956079+0000 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/1771523969 expires 2024-04-26T17:01:02.956079+0000 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/1771523969 expires 2024-04-26T17:01:02.956079+0000 2024-04-25T17:10:02.490 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2952856121 expires 2024-04-26T17:01:02.956079+0000 2024-04-25T17:10:02.491 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/548635250 expires 2024-04-26T17:00:35.573967+0000 2024-04-25T17:10:02.491 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/3206171750 expires 2024-04-26T17:00:35.573967+0000 2024-04-25T17:10:02.491 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/4227184783 expires 2024-04-26T17:00:35.573967+0000 2024-04-25T17:10:02.491 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2705351311 expires 2024-04-26T17:00:35.573967+0000 2024-04-25T17:10:02.491 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/4227184783 expires 2024-04-26T17:00:35.573967+0000 2024-04-25T17:10:02.491 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/806329858 expires 2024-04-26T17:01:02.956079+0000 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0c0c0afb80 msgr2=0x7f0c0c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0c0c0afb80 0x7f0c0c0b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0c20104110 tx=0x7f0c1c009190 comp rx=0 tx=0).stop 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104210 msgr2=0x7f0c2019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104210 0x7f0c2019f6a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000b530 tx=0x7f0c1000b840 comp rx=0 tx=0).stop 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 shutdown_connections 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c20104bb0 0x7f0c201a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c201030e0 0x7f0c20076320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0c0c0afb80 0x7f0c0c0b2090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 --2- 172.21.15.96:0/4041992168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c20104210 0x7f0c2019f6a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 >> 172.21.15.96:0/4041992168 conn(0x7f0c200fe740 msgr2=0x7f0c20107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 shutdown_connections 2024-04-25T17:10:02.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:02.490+0000 7f0c279d7700 1 -- 172.21.15.96:0/4041992168 wait complete. 2024-04-25T17:10:02.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:02 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/4041992168' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T17:10:02.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:02 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/4041992168' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T17:10:03.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:02 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/4041992168' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T17:10:03.239 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T17:10:03.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:10:03.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 -- 172.21.15.96:0/1521010339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40102fb0 msgr2=0x7f2c401033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/1521010339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40102fb0 0x7f2c401033c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009ca0 tx=0x7f2c3802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 -- 172.21.15.96:0/1521010339 shutdown_connections 2024-04-25T17:10:03.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/1521010339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c40104670 0x7f2c4010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/1521010339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c40103b10 0x7f2c40103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/1521010339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40102fb0 0x7f2c401033c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 -- 172.21.15.96:0/1521010339 >> 172.21.15.96:0/1521010339 conn(0x7f2c400fd330 msgr2=0x7f2c400ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:03.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 -- 172.21.15.96:0/1521010339 shutdown_connections 2024-04-25T17:10:03.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.362+0000 7f2c46cf9640 1 -- 172.21.15.96:0/1521010339 wait complete. 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 Processor -- start 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 -- start start 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c40102fb0 0x7f2c40102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40103b10 0x7f2c4019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c4019f380 0x7f2c401a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4010e710 con 0x7f2c40103b10 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4010e590 con 0x7f2c40102fb0 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c46cf9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4010e890 con 0x7f2c4019f380 2024-04-25T17:10:03.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.363+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40103b10 0x7f2c4019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40103b10 0x7f2c4019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57930/0 (socket says 172.21.15.96:57930) 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 -- 172.21.15.96:0/403240665 learned_addr learned my addr 172.21.15.96:0/403240665 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c44a95640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c40102fb0 0x7f2c40102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 -- 172.21.15.96:0/403240665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c4019f380 msgr2=0x7f2c401a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c4019f380 0x7f2c401a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 -- 172.21.15.96:0/403240665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c40102fb0 msgr2=0x7f2c40102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c40102fb0 0x7f2c40102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 -- 172.21.15.96:0/403240665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c401a1f70 con 0x7f2c40103b10 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3ffff640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40103b10 0x7f2c4019ee40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800d950 tx=0x7f2c2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3dffb640 1 -- 172.21.15.96:0/403240665 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28017070 con 0x7f2c40103b10 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3dffb640 1 -- 172.21.15.96:0/403240665 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c28009c30 con 0x7f2c40103b10 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c401a2260 con 0x7f2c40103b10 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c3dffb640 1 -- 172.21.15.96:0/403240665 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800f620 con 0x7f2c40103b10 2024-04-25T17:10:03.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.364+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401a27b0 con 0x7f2c40103b10 2024-04-25T17:10:03.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.365+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4006b400 con 0x7f2c40103b10 2024-04-25T17:10:03.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.366+0000 7f2c3dffb640 1 -- 172.21.15.96:0/403240665 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2c28010460 con 0x7f2c40103b10 2024-04-25T17:10:03.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.367+0000 7f2c3dffb640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2c2c0afaf0 0x7f2c2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.367+0000 7f2c44a95640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2c2c0afaf0 0x7f2c2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.367+0000 7f2c3dffb640 1 -- 172.21.15.96:0/403240665 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f2c280de890 con 0x7f2c40103b10 2024-04-25T17:10:03.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.368+0000 7f2c44a95640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2c2c0afaf0 0x7f2c2c0b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2c380094e0 tx=0x7f2c38004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.372+0000 7f2c3dffb640 1 -- 172.21.15.96:0/403240665 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c280a7470 con 0x7f2c40103b10 2024-04-25T17:10:03.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.554+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c4002def0 con 0x7f2c40103b10 2024-04-25T17:10:03.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.556+0000 7f2c3dffb640 1 -- 172.21.15.96:0/403240665 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22594 (secure 0 0 0) 0x7f2c28009de0 con 0x7f2c40103b10 2024-04-25T17:10:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:10:03.556 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:09:04.141423+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2c2c0afaf0 msgr2=0x7f2c2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2c2c0afaf0 0x7f2c2c0b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2c380094e0 tx=0x7f2c38004a40 comp rx=0 tx=0).stop 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40103b10 msgr2=0x7f2c4019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40103b10 0x7f2c4019ee40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800d950 tx=0x7f2c2800de10 comp rx=0 tx=0).stop 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 shutdown_connections 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f2c2c0afaf0 0x7f2c2c0b2000 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c4019f380 0x7f2c401a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c40103b10 0x7f2c4019ee40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 --2- 172.21.15.96:0/403240665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c40102fb0 0x7f2c40102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 >> 172.21.15.96:0/403240665 conn(0x7f2c400fd330 msgr2=0x7f2c400ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:03.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 shutdown_connections 2024-04-25T17:10:03.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.559+0000 7f2c46cf9640 1 -- 172.21.15.96:0/403240665 wait complete. 2024-04-25T17:10:03.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T17:08:09.266201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T17:08:10.452964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:10:03.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:10:03.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 -- 172.21.15.96:0/3732318227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc109a30 msgr2=0x7f7fdc10be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/3732318227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc109a30 0x7f7fdc10be50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009ca0 tx=0x7f7fd402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 -- 172.21.15.96:0/3732318227 shutdown_connections 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/3732318227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc109a30 0x7f7fdc10be50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/3732318227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fdc104d30 0x7f7fdc105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/3732318227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fdc103af0 0x7f7fdc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 -- 172.21.15.96:0/3732318227 >> 172.21.15.96:0/3732318227 conn(0x7f7fdc0fece0 msgr2=0x7f7fdc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 -- 172.21.15.96:0/3732318227 shutdown_connections 2024-04-25T17:10:03.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.679+0000 7f7fe0c92640 1 -- 172.21.15.96:0/3732318227 wait complete. 2024-04-25T17:10:03.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 Processor -- start 2024-04-25T17:10:03.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 -- start start 2024-04-25T17:10:03.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fdc103af0 0x7f7fdc19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc104d30 0x7f7fdc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fdc109a30 0x7f7fdc1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10f620 con 0x7f7fdc104d30 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10f4a0 con 0x7f7fdc109a30 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fe0c92640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10f7a0 con 0x7f7fdc103af0 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc104d30 0x7f7fdc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc104d30 0x7f7fdc19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57938/0 (socket says 172.21.15.96:57938) 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fd3fff640 1 -- 172.21.15.96:0/2437547743 learned_addr learned my addr 172.21.15.96:0/2437547743 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.680+0000 7f7fdad9d640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fdc109a30 0x7f7fdc1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fda59c640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fdc103af0 0x7f7fdc19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd3fff640 1 -- 172.21.15.96:0/2437547743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fdc103af0 msgr2=0x7f7fdc19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd3fff640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fdc103af0 0x7f7fdc19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd3fff640 1 -- 172.21.15.96:0/2437547743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fdc109a30 msgr2=0x7f7fdc1a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd3fff640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fdc109a30 0x7f7fdc1a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd3fff640 1 -- 172.21.15.96:0/2437547743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc1a1d10 con 0x7f7fdc104d30 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fdad9d640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fdc109a30 0x7f7fdc1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd3fff640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc104d30 0x7f7fdc19a710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400d770 tx=0x7f7fc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd37fe640 1 -- 172.21.15.96:0/2437547743 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4017070 con 0x7f7fdc104d30 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd37fe640 1 -- 172.21.15.96:0/2437547743 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc40054d0 con 0x7f7fdc104d30 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc1a1fa0 con 0x7f7fdc104d30 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fd37fe640 1 -- 172.21.15.96:0/2437547743 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4004c30 con 0x7f7fdc104d30 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.681+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc1a2440 con 0x7f7fdc104d30 2024-04-25T17:10:03.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.682+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fdc108840 con 0x7f7fdc104d30 2024-04-25T17:10:03.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.683+0000 7f7fd37fe640 1 -- 172.21.15.96:0/2437547743 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7fc4004dd0 con 0x7f7fdc104d30 2024-04-25T17:10:03.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.684+0000 7f7fd37fe640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7fc80afaf0 0x7f7fc80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.684+0000 7f7fda59c640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7fc80afaf0 0x7f7fc80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.684+0000 7f7fd37fe640 1 -- 172.21.15.96:0/2437547743 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f7fc40deaf0 con 0x7f7fdc104d30 2024-04-25T17:10:03.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.684+0000 7f7fda59c640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7fc80afaf0 0x7f7fc80b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005d00 tx=0x7f7fcc005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:03.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.688+0000 7f7fd37fe640 1 -- 172.21.15.96:0/2437547743 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fc40a76d0 con 0x7f7fdc104d30 2024-04-25T17:10:03.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.865+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7fdc02dce0 con 0x7f7fdc104d30 2024-04-25T17:10:03.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.865+0000 7f7fd37fe640 1 -- 172.21.15.96:0/2437547743 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f7fc40a6fd0 con 0x7f7fdc104d30 2024-04-25T17:10:03.865 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7fc80afaf0 msgr2=0x7f7fc80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7fc80afaf0 0x7f7fc80b2000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005d00 tx=0x7f7fcc005c50 comp rx=0 tx=0).stop 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc104d30 msgr2=0x7f7fdc19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc104d30 0x7f7fdc19a710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400d770 tx=0x7f7fc400dc30 comp rx=0 tx=0).stop 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 shutdown_connections 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fdc109a30 0x7f7fdc1a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fdc104d30 0x7f7fdc19a710 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7fc80afaf0 0x7f7fc80b2000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 --2- 172.21.15.96:0/2437547743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fdc103af0 0x7f7fdc19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 >> 172.21.15.96:0/2437547743 conn(0x7f7fdc0fece0 msgr2=0x7f7fdc105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 shutdown_connections 2024-04-25T17:10:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.868+0000 7f7fe0c92640 1 -- 172.21.15.96:0/2437547743 wait complete. 2024-04-25T17:10:03.878 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:03 smithi096 ceph-mon[29131]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:03.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T17:10:03.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:03 smithi097 ceph-mon[32602]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 -- 172.21.15.96:0/2564617481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094109890 msgr2=0x7fc09410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/2564617481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094109890 0x7fc09410bcb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009900 tx=0x7fc08c02a410 comp rx=0 tx=0).stop 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 -- 172.21.15.96:0/2564617481 shutdown_connections 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/2564617481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094109890 0x7fc09410bcb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/2564617481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc094106f30 0x7fc094109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/2564617481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0941045d0 0x7fc0941069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 -- 172.21.15.96:0/2564617481 >> 172.21.15.96:0/2564617481 conn(0x7fc0940fdbd0 msgr2=0x7fc094100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 -- 172.21.15.96:0/2564617481 shutdown_connections 2024-04-25T17:10:03.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 -- 172.21.15.96:0/2564617481 wait complete. 2024-04-25T17:10:03.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.990+0000 7fc09a1ee640 1 Processor -- start 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 -- start start 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0941045d0 0x7fc09419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094106f30 0x7fc09419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc094109890 0x7fc0941a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410f610 con 0x7fc094106f30 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410f490 con 0x7fc0941045d0 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410f790 con 0x7fc094109890 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094106f30 0x7fc09419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094106f30 0x7fc09419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57960/0 (socket says 172.21.15.96:57960) 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 -- 172.21.15.96:0/3803630647 learned_addr learned my addr 172.21.15.96:0/3803630647 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc093fff640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc094109890 0x7fc0941a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc0937fe640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0941045d0 0x7fc09419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 -- 172.21.15.96:0/3803630647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc094109890 msgr2=0x7fc0941a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc094109890 0x7fc0941a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 -- 172.21.15.96:0/3803630647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0941045d0 msgr2=0x7fc09419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0941045d0 0x7fc09419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 -- 172.21.15.96:0/3803630647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0941a2080 con 0x7fc094106f30 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc0937fe640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0941045d0 0x7fc09419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc092ffd640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094106f30 0x7fc09419aa80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc08800ea80 tx=0x7fc08800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc090ff9640 1 -- 172.21.15.96:0/3803630647 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08800ce40 con 0x7fc094106f30 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0941a2370 con 0x7fc094106f30 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc090ff9640 1 -- 172.21.15.96:0/3803630647 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc088005020 con 0x7fc094106f30 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc090ff9640 1 -- 172.21.15.96:0/3803630647 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880116d0 con 0x7fc094106f30 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.991+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0941a28c0 con 0x7fc094106f30 2024-04-25T17:10:03.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.992+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09406c2a0 con 0x7fc094106f30 2024-04-25T17:10:03.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.994+0000 7fc090ff9640 1 -- 172.21.15.96:0/3803630647 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc0880054d0 con 0x7fc094106f30 2024-04-25T17:10:03.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.995+0000 7fc090ff9640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc07c0afaf0 0x7fc07c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:03.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.995+0000 7fc0937fe640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc07c0afaf0 0x7fc07c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:03.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.995+0000 7fc090ff9640 1 -- 172.21.15.96:0/3803630647 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fc0880df370 con 0x7fc094106f30 2024-04-25T17:10:03.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.995+0000 7fc0937fe640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc07c0afaf0 0x7fc07c0b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc09406a8b0 tx=0x7fc084005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:03.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:03.999+0000 7fc090ff9640 1 -- 172.21.15.96:0/3803630647 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc088016030 con 0x7fc094106f30 2024-04-25T17:10:04.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:03 smithi171 ceph-mon[33189]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:04.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.176+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc0941a2e40 con 0x7fc094106f30 2024-04-25T17:10:04.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.177+0000 7fc090ff9640 1 -- 172.21.15.96:0/3803630647 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7fc0880a7f50 con 0x7fc094106f30 2024-04-25T17:10:04.177 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc07c0afaf0 msgr2=0x7fc07c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc07c0afaf0 0x7fc07c0b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc09406a8b0 tx=0x7fc084005f90 comp rx=0 tx=0).stop 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094106f30 msgr2=0x7fc09419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094106f30 0x7fc09419aa80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc08800ea80 tx=0x7fc08800ef40 comp rx=0 tx=0).stop 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 shutdown_connections 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc07c0afaf0 0x7fc07c0b2000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc094109890 0x7fc0941a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc094106f30 0x7fc09419aa80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 --2- 172.21.15.96:0/3803630647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0941045d0 0x7fc09419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 >> 172.21.15.96:0/3803630647 conn(0x7fc0940fdbd0 msgr2=0x7fc094105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:04.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 shutdown_connections 2024-04-25T17:10:04.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.180+0000 7fc09a1ee640 1 -- 172.21.15.96:0/3803630647 wait complete. 2024-04-25T17:10:04.190 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T17:10:04.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 -- 172.21.15.96:0/1450635459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1033d0 msgr2=0x7fb3dc1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/1450635459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1033d0 0x7fb3dc1037e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009900 tx=0x7fb3c802a410 comp rx=0 tx=0).stop 2024-04-25T17:10:04.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 -- 172.21.15.96:0/1450635459 shutdown_connections 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/1450635459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3dc1080d0 0x7fb3dc10a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/1450635459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1033d0 0x7fb3dc1037e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/1450635459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc102190 0x7fb3dc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 -- 172.21.15.96:0/1450635459 >> 172.21.15.96:0/1450635459 conn(0x7fb3dc0fd380 msgr2=0x7fb3dc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 -- 172.21.15.96:0/1450635459 shutdown_connections 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.301+0000 7fb3e2a96640 1 -- 172.21.15.96:0/1450635459 wait complete. 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 Processor -- start 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 -- start start 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc102190 0x7fb3dc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3dc1033d0 0x7fb3dc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1080d0 0x7fb3dc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10dcb0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10db30 con 0x7fb3dc1033d0 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e2a96640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10de30 con 0x7fb3dc102190 2024-04-25T17:10:04.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1080d0 0x7fb3dc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1080d0 0x7fb3dc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57990/0 (socket says 172.21.15.96:57990) 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 -- 172.21.15.96:0/686610404 learned_addr learned my addr 172.21.15.96:0/686610404 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e0832640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc102190 0x7fb3dc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 -- 172.21.15.96:0/686610404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc102190 msgr2=0x7fb3dc199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc102190 0x7fb3dc199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 -- 172.21.15.96:0/686610404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3dc1033d0 msgr2=0x7fb3dc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3dbfff640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3dc1033d0 0x7fb3dc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3dc1033d0 0x7fb3dc1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.302+0000 7fb3e1033640 1 -- 172.21.15.96:0/686610404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc1a0cb0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.303+0000 7fb3e1033640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1080d0 0x7fb3dc1a0640 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400eaa0 tx=0x7fb3d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.303+0000 7fb3d9ffb640 1 -- 172.21.15.96:0/686610404 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d400cdb0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.303+0000 7fb3d9ffb640 1 -- 172.21.15.96:0/686610404 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d4005020 con 0x7fb3dc1080d0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.303+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc1a0fa0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.303+0000 7fb3d9ffb640 1 -- 172.21.15.96:0/686610404 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40116d0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.303+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc1a13b0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.304+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3dc06a160 con 0x7fb3dc1080d0 2024-04-25T17:10:04.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.305+0000 7fb3d9ffb640 1 -- 172.21.15.96:0/686610404 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb3d40051d0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.305+0000 7fb3d9ffb640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb3cc0afaf0 0x7fb3cc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.306+0000 7fb3e0832640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb3cc0afaf0 0x7fb3cc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.306+0000 7fb3d9ffb640 1 -- 172.21.15.96:0/686610404 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fb3d40dee40 con 0x7fb3dc1080d0 2024-04-25T17:10:04.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.306+0000 7fb3e0832640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb3cc0afaf0 0x7fb3cc0b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005fd0 tx=0x7fb3d000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:04.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.310+0000 7fb3d9ffb640 1 -- 172.21.15.96:0/686610404 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3d4016030 con 0x7fb3dc1080d0 2024-04-25T17:10:04.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.489+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb3dc0637a0 con 0x7fb3dc1080d0 2024-04-25T17:10:04.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.490+0000 7fb3d9ffb640 1 -- 172.21.15.96:0/686610404 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7fb3d40a7a20 con 0x7fb3dc1080d0 2024-04-25T17:10:04.490 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb3cc0afaf0 msgr2=0x7fb3cc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb3cc0afaf0 0x7fb3cc0b2000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005fd0 tx=0x7fb3d000b390 comp rx=0 tx=0).stop 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1080d0 msgr2=0x7fb3dc1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1080d0 0x7fb3dc1a0640 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400eaa0 tx=0x7fb3d400ef60 comp rx=0 tx=0).stop 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 shutdown_connections 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3dc1080d0 0x7fb3dc1a0640 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3dc1033d0 0x7fb3dc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fb3cc0afaf0 0x7fb3cc0b2000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 --2- 172.21.15.96:0/686610404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb3dc102190 0x7fb3dc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 >> 172.21.15.96:0/686610404 conn(0x7fb3dc0fd380 msgr2=0x7fb3dc0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 shutdown_connections 2024-04-25T17:10:04.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.492+0000 7fb3e2a96640 1 -- 172.21.15.96:0/686610404 wait complete. 2024-04-25T17:10:04.503 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:10:04.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 -- 172.21.15.96:0/1415333415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 msgr2=0x7f4d1c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1415333415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c10be60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009900 tx=0x7f4d0c02a410 comp rx=0 tx=0).stop 2024-04-25T17:10:04.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 -- 172.21.15.96:0/1415333415 shutdown_connections 2024-04-25T17:10:04.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1415333415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c10be60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1415333415 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d1c104d40 0x7f4d1c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1415333415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d1c103b00 0x7f4d1c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 -- 172.21.15.96:0/1415333415 >> 172.21.15.96:0/1415333415 conn(0x7f4d1c0fed30 msgr2=0x7f4d1c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:04.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 -- 172.21.15.96:0/1415333415 shutdown_connections 2024-04-25T17:10:04.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 -- 172.21.15.96:0/1415333415 wait complete. 2024-04-25T17:10:04.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.611+0000 7f4d2179d640 1 Processor -- start 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d2179d640 1 -- start start 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d2179d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d1c103b00 0x7f4d1c19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d2179d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d1c104d40 0x7f4d1c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d2179d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d2179d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10f680 con 0x7f4d1c109a40 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d2179d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10f500 con 0x7f4d1c103b00 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d2179d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10f800 con 0x7f4d1c104d40 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c1a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58020/0 (socket says 172.21.15.96:58020) 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 -- 172.21.15.96:0/1022196129 learned_addr learned my addr 172.21.15.96:0/1022196129 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1a7fc640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d1c104d40 0x7f4d1c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1affd640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d1c103b00 0x7f4d1c19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 -- 172.21.15.96:0/1022196129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d1c104d40 msgr2=0x7f4d1c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d1c104d40 0x7f4d1c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 -- 172.21.15.96:0/1022196129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d1c103b00 msgr2=0x7f4d1c19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d1c103b00 0x7f4d1c19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 -- 172.21.15.96:0/1022196129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c1a1d30 con 0x7f4d1c109a40 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1affd640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d1c103b00 0x7f4d1c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.612+0000 7f4d1b7fe640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c1a16c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c005ab0 tx=0x7f4d0c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.613+0000 7f4d03fff640 1 -- 172.21.15.96:0/1022196129 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c03f030 con 0x7f4d1c109a40 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.613+0000 7f4d03fff640 1 -- 172.21.15.96:0/1022196129 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d0c003cb0 con 0x7f4d1c109a40 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.613+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1a1fc0 con 0x7f4d1c109a40 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.613+0000 7f4d03fff640 1 -- 172.21.15.96:0/1022196129 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c035070 con 0x7f4d1c109a40 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.613+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1a24a0 con 0x7f4d1c109a40 2024-04-25T17:10:04.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.614+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1c108850 con 0x7f4d1c109a40 2024-04-25T17:10:04.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.615+0000 7f4d03fff640 1 -- 172.21.15.96:0/1022196129 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4d0c042030 con 0x7f4d1c109a40 2024-04-25T17:10:04.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.616+0000 7f4d03fff640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d040afb70 0x7f4d040b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.616+0000 7f4d1affd640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d040afb70 0x7f4d040b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.616+0000 7f4d03fff640 1 -- 172.21.15.96:0/1022196129 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f4d0c0ff7f0 con 0x7f4d1c109a40 2024-04-25T17:10:04.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.616+0000 7f4d1affd640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d040afb70 0x7f4d040b2080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c104700 tx=0x7f4d1400a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:04.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.620+0000 7f4d03fff640 1 -- 172.21.15.96:0/1022196129 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d0c0c83d0 con 0x7f4d1c109a40 2024-04-25T17:10:04.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.839+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d1c02dc70 con 0x7f4d1c109a40 2024-04-25T17:10:04.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.840+0000 7f4d03fff640 1 -- 172.21.15.96:0/1022196129 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11805 (secure 0 0 0) 0x7f4d0c0c7cd0 con 0x7f4d1c109a40 2024-04-25T17:10:04.839 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:10:04.839 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:08:14.735284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:10:04.839 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d040afb70 msgr2=0x7f4d040b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d040afb70 0x7f4d040b2080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c104700 tx=0x7f4d1400a560 comp rx=0 tx=0).stop 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 msgr2=0x7f4d1c1a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c1a16c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c005ab0 tx=0x7f4d0c004360 comp rx=0 tx=0).stop 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 shutdown_connections 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1c109a40 0x7f4d1c1a16c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f4d040afb70 0x7f4d040b2080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d1c104d40 0x7f4d1c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 --2- 172.21.15.96:0/1022196129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d1c103b00 0x7f4d1c19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 >> 172.21.15.96:0/1022196129 conn(0x7f4d1c0fed30 msgr2=0x7f4d1c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:04.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 shutdown_connections 2024-04-25T17:10:04.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.842+0000 7f4d2179d640 1 -- 172.21.15.96:0/1022196129 wait complete. 2024-04-25T17:10:04.851 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:04 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/403240665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:04.851 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:04 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2437547743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:10:04.851 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:04 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3803630647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:10:04.852 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:04 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/686610404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:10:04.853 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 -- 172.21.15.96:0/3190762176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c78103c00 msgr2=0x7f3c78104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 --2- 172.21.15.96:0/3190762176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c78103c00 0x7f3c78104010 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009900 tx=0x7f3c6c02a410 comp rx=0 tx=0).stop 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 -- 172.21.15.96:0/3190762176 shutdown_connections 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 --2- 172.21.15.96:0/3190762176 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78108900 0x7f3c7810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 --2- 172.21.15.96:0/3190762176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c78103c00 0x7f3c78104010 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 --2- 172.21.15.96:0/3190762176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c781029c0 0x7f3c781036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 -- 172.21.15.96:0/3190762176 >> 172.21.15.96:0/3190762176 conn(0x7f3c780fdbd0 msgr2=0x7f3c78100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 -- 172.21.15.96:0/3190762176 shutdown_connections 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 -- 172.21.15.96:0/3190762176 wait complete. 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 Processor -- start 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.962+0000 7f3c7d584640 1 -- start start 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c781029c0 0x7f3c78199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c78103c00 0x7f3c78199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78108900 0x7f3c781a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810e4f0 con 0x7f3c781029c0 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810e370 con 0x7f3c78103c00 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7810e670 con 0x7f3c78108900 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c781029c0 0x7f3c78199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c781029c0 0x7f3c78199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58036/0 (socket says 172.21.15.96:58036) 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 -- 172.21.15.96:0/2097965791 learned_addr learned my addr 172.21.15.96:0/2097965791 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c767fc640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c78103c00 0x7f3c78199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c777fe640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78108900 0x7f3c781a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 -- 172.21.15.96:0/2097965791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78108900 msgr2=0x7f3c781a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78108900 0x7f3c781a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 -- 172.21.15.96:0/2097965791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c78103c00 msgr2=0x7f3c78199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c78103c00 0x7f3c78199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 -- 172.21.15.96:0/2097965791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c781a0b90 con 0x7f3c781029c0 2024-04-25T17:10:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c767fc640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c78103c00 0x7f3c78199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:04.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c76ffd640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c781029c0 0x7f3c78199100 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800cce0 tx=0x7f3c6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:04.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c5ffff640 1 -- 172.21.15.96:0/2097965791 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800f450 con 0x7f3c781029c0 2024-04-25T17:10:04.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c781a0e80 con 0x7f3c781029c0 2024-04-25T17:10:04.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c5ffff640 1 -- 172.21.15.96:0/2097965791 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6800faa0 con 0x7f3c781029c0 2024-04-25T17:10:04.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c5ffff640 1 -- 172.21.15.96:0/2097965791 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68018ae0 con 0x7f3c781029c0 2024-04-25T17:10:04.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.963+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c781a12e0 con 0x7f3c781029c0 2024-04-25T17:10:04.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.964+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7806afa0 con 0x7f3c781029c0 2024-04-25T17:10:04.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.966+0000 7f3c5ffff640 1 -- 172.21.15.96:0/2097965791 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f3c68018c80 con 0x7f3c781029c0 2024-04-25T17:10:04.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.966+0000 7f3c5ffff640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3c600afaf0 0x7f3c600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:04.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.966+0000 7f3c767fc640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3c600afaf0 0x7f3c600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:04.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.966+0000 7f3c5ffff640 1 -- 172.21.15.96:0/2097965791 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f3c680df210 con 0x7f3c781029c0 2024-04-25T17:10:04.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.967+0000 7f3c767fc640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3c600afaf0 0x7f3c600b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c000c00 tx=0x7f3c6c005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:04.970+0000 7f3c5ffff640 1 -- 172.21.15.96:0/2097965791 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c68052ff0 con 0x7f3c781029c0 2024-04-25T17:10:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:04 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/403240665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:04 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2437547743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:10:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:04 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3803630647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:10:04.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:04 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/686610404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:10:05.029 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:04 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/403240665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:04 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2437547743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:10:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:04 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3803630647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:10:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:04 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/686610404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:10:05.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.150+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c7802dc10 con 0x7f3c781029c0 2024-04-25T17:10:05.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.152+0000 7f3c5ffff640 1 -- 172.21.15.96:0/2097965791 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22594 (secure 0 0 0) 0x7f3c680aee20 con 0x7f3c781029c0 2024-04-25T17:10:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:10:05.151 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:09:04.141423+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:10:05.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3c600afaf0 msgr2=0x7f3c600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3c600afaf0 0x7f3c600b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c000c00 tx=0x7f3c6c005d50 comp rx=0 tx=0).stop 2024-04-25T17:10:05.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c781029c0 msgr2=0x7f3c78199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c781029c0 0x7f3c78199100 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800cce0 tx=0x7f3c6800a620 comp rx=0 tx=0).stop 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 shutdown_connections 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f3c600afaf0 0x7f3c600b2000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c78108900 0x7f3c781a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c78103c00 0x7f3c78199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 --2- 172.21.15.96:0/2097965791 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c781029c0 0x7f3c78199100 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 >> 172.21.15.96:0/2097965791 conn(0x7f3c780fdbd0 msgr2=0x7f3c781048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 shutdown_connections 2024-04-25T17:10:05.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.154+0000 7f3c7d584640 1 -- 172.21.15.96:0/2097965791 wait complete. 2024-04-25T17:10:05.165 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T17:08:09.266201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T17:08:10.452964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:10:05.165 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.276+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/2104743032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68103c30 msgr2=0x7f0b68104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.276+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/2104743032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68103c30 0x7f0b68104040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009900 tx=0x7f0b5c02a410 comp rx=0 tx=0).stop 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/2104743032 shutdown_connections 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/2104743032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b68108930 0x7f0b6810ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/2104743032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68103c30 0x7f0b68104040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/2104743032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b681029f0 0x7f0b681036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/2104743032 >> 172.21.15.96:0/2104743032 conn(0x7f0b680fdc00 msgr2=0x7f0b68100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/2104743032 shutdown_connections 2024-04-25T17:10:05.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/2104743032 wait complete. 2024-04-25T17:10:05.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 Processor -- start 2024-04-25T17:10:05.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.277+0000 7f0b6d9c0640 1 -- start start 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b681029f0 0x7f0b6806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b68103c30 0x7f0b6819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68108930 0x7f0b681a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6810e510 con 0x7f0b68108930 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6810e390 con 0x7f0b681029f0 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6810e690 con 0x7f0b68103c30 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68108930 0x7f0b681a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68108930 0x7f0b681a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58062/0 (socket says 172.21.15.96:58062) 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 -- 172.21.15.96:0/1503469909 learned_addr learned my addr 172.21.15.96:0/1503469909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b66ffd640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b681029f0 0x7f0b6806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b667fc640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b68103c30 0x7f0b6819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 -- 172.21.15.96:0/1503469909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b68103c30 msgr2=0x7f0b6819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b68103c30 0x7f0b6819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 -- 172.21.15.96:0/1503469909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b681029f0 msgr2=0x7f0b6806bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b681029f0 0x7f0b6806bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 -- 172.21.15.96:0/1503469909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b681a1d60 con 0x7f0b68108930 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b677fe640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68108930 0x7f0b681a1540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000eaa0 tx=0x7f0b6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6c9be640 1 -- 172.21.15.96:0/1503469909 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6000cdb0 con 0x7f0b68108930 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b681a1ff0 con 0x7f0b68108930 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6c9be640 1 -- 172.21.15.96:0/1503469909 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b60005020 con 0x7f0b68108930 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6c9be640 1 -- 172.21.15.96:0/1503469909 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b600116d0 con 0x7f0b68108930 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.278+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681a24b0 con 0x7f0b68108930 2024-04-25T17:10:05.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.279+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b68107740 con 0x7f0b68108930 2024-04-25T17:10:05.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.281+0000 7f0b6c9be640 1 -- 172.21.15.96:0/1503469909 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0b600051d0 con 0x7f0b68108930 2024-04-25T17:10:05.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.281+0000 7f0b6c9be640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0b500afaf0 0x7f0b500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.281+0000 7f0b66ffd640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0b500afaf0 0x7f0b500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.281+0000 7f0b6c9be640 1 -- 172.21.15.96:0/1503469909 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f0b600df0d0 con 0x7f0b68108930 2024-04-25T17:10:05.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.282+0000 7f0b66ffd640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0b500afaf0 0x7f0b500b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b58005fd0 tx=0x7f0b5800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:05.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.285+0000 7f0b6c9be640 1 -- 172.21.15.96:0/1503469909 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b60016030 con 0x7f0b68108930 2024-04-25T17:10:05.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.461+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b680637a0 con 0x7f0b68108930 2024-04-25T17:10:05.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.461+0000 7f0b6c9be640 1 -- 172.21.15.96:0/1503469909 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f0b600a7660 con 0x7f0b68108930 2024-04-25T17:10:05.460 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:10:05.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0b500afaf0 msgr2=0x7f0b500b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0b500afaf0 0x7f0b500b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b58005fd0 tx=0x7f0b5800b390 comp rx=0 tx=0).stop 2024-04-25T17:10:05.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68108930 msgr2=0x7f0b681a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68108930 0x7f0b681a1540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000eaa0 tx=0x7f0b6000ef60 comp rx=0 tx=0).stop 2024-04-25T17:10:05.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 shutdown_connections 2024-04-25T17:10:05.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b68108930 0x7f0b681a1540 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f0b500afaf0 0x7f0b500b2000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b68103c30 0x7f0b6819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 --2- 172.21.15.96:0/1503469909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b681029f0 0x7f0b6806bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 >> 172.21.15.96:0/1503469909 conn(0x7f0b680fdc00 msgr2=0x7f0b68104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:05.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 shutdown_connections 2024-04-25T17:10:05.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.463+0000 7f0b6d9c0640 1 -- 172.21.15.96:0/1503469909 wait complete. 2024-04-25T17:10:05.473 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T17:10:05.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 -- 172.21.15.96:0/2181581732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd0101760 msgr2=0x7f7cd0101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 --2- 172.21.15.96:0/2181581732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd0101760 0x7f7cd0101bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009900 tx=0x7f7cc402a410 comp rx=0 tx=0).stop 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 -- 172.21.15.96:0/2181581732 shutdown_connections 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 --2- 172.21.15.96:0/2181581732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd0102260 0x7f7cd010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 --2- 172.21.15.96:0/2181581732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd0101760 0x7f7cd0101bd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 --2- 172.21.15.96:0/2181581732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd01050b0 0x7f7cd01054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 -- 172.21.15.96:0/2181581732 >> 172.21.15.96:0/2181581732 conn(0x7f7cd00fd380 msgr2=0x7f7cd00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 -- 172.21.15.96:0/2181581732 shutdown_connections 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.585+0000 7f7cd536b640 1 -- 172.21.15.96:0/2181581732 wait complete. 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 Processor -- start 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 -- start start 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd0101760 0x7f7cd019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd0102260 0x7f7cd019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd01050b0 0x7f7cd01a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd010f620 con 0x7f7cd01050b0 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd010f4a0 con 0x7f7cd0102260 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.586+0000 7f7cd536b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd010f7a0 con 0x7f7cd0101760 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd01050b0 0x7f7cd01a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd01050b0 0x7f7cd01a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58082/0 (socket says 172.21.15.96:58082) 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 -- 172.21.15.96:0/410169574 learned_addr learned my addr 172.21.15.96:0/410169574 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7cce7fc640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd0102260 0x7f7cd019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 -- 172.21.15.96:0/410169574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd0101760 msgr2=0x7f7cd019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd0101760 0x7f7cd019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 -- 172.21.15.96:0/410169574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd0102260 msgr2=0x7f7cd019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd0102260 0x7f7cd019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 -- 172.21.15.96:0/410169574 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd01a1ce0 con 0x7f7cd01050b0 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7cce7fc640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd0102260 0x7f7cd019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7ccf7fe640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd01050b0 0x7f7cd01a1720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800c970 tx=0x7f7cc800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7cb7fff640 1 -- 172.21.15.96:0/410169574 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8007bb0 con 0x7f7cd01050b0 2024-04-25T17:10:05.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7cb7fff640 1 -- 172.21.15.96:0/410169574 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8007d50 con 0x7f7cd01050b0 2024-04-25T17:10:05.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd01a1fd0 con 0x7f7cd01050b0 2024-04-25T17:10:05.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7cb7fff640 1 -- 172.21.15.96:0/410169574 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8018770 con 0x7f7cd01050b0 2024-04-25T17:10:05.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.587+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd01a23e0 con 0x7f7cd01050b0 2024-04-25T17:10:05.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.588+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd006a160 con 0x7f7cd01050b0 2024-04-25T17:10:05.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.590+0000 7f7cb7fff640 1 -- 172.21.15.96:0/410169574 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7cc800f990 con 0x7f7cd01050b0 2024-04-25T17:10:05.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.590+0000 7f7cb7fff640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7cb80afaf0 0x7f7cb80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.590+0000 7f7cceffd640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7cb80afaf0 0x7f7cb80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.590+0000 7f7cb7fff640 1 -- 172.21.15.96:0/410169574 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f7cc80dee60 con 0x7f7cd01050b0 2024-04-25T17:10:05.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.591+0000 7f7cceffd640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7cb80afaf0 0x7f7cb80b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006b2d0 tx=0x7f7cc00093c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:05.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.594+0000 7f7cb7fff640 1 -- 172.21.15.96:0/410169574 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cc8014030 con 0x7f7cd01050b0 2024-04-25T17:10:05.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.771+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7cd002db50 con 0x7f7cd01050b0 2024-04-25T17:10:05.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.771+0000 7f7cb7fff640 1 -- 172.21.15.96:0/410169574 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7f7cc80a7ad0 con 0x7f7cd01050b0 2024-04-25T17:10:05.770 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T17:10:05.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7cb80afaf0 msgr2=0x7f7cb80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7cb80afaf0 0x7f7cb80b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006b2d0 tx=0x7f7cc00093c0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd01050b0 msgr2=0x7f7cd01a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd01050b0 0x7f7cd01a1720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800c970 tx=0x7f7cc800ce30 comp rx=0 tx=0).stop 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 shutdown_connections 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd01050b0 0x7f7cd01a1720 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd0102260 0x7f7cd019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f7cb80afaf0 0x7f7cb80b2000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 --2- 172.21.15.96:0/410169574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd0101760 0x7f7cd019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 >> 172.21.15.96:0/410169574 conn(0x7f7cd00fd380 msgr2=0x7f7cd0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 shutdown_connections 2024-04-25T17:10:05.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.773+0000 7f7cd536b640 1 -- 172.21.15.96:0/410169574 wait complete. 2024-04-25T17:10:05.782 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:05 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1022196129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:10:05.783 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:05 smithi096 ceph-mon[29131]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:10:05.783 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:05 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2097965791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:05.783 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:05 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1503469909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:10:05.784 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.894+0000 7fae77f84640 1 -- 172.21.15.96:0/2704139044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70104460 msgr2=0x7fae7010abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.894+0000 7fae77f84640 1 --2- 172.21.15.96:0/2704139044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70104460 0x7fae7010abb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae68009900 tx=0x7fae6802a410 comp rx=0 tx=0).stop 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 -- 172.21.15.96:0/2704139044 shutdown_connections 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 --2- 172.21.15.96:0/2704139044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70104460 0x7fae7010abb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 --2- 172.21.15.96:0/2704139044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae70103940 0x7fae70103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 --2- 172.21.15.96:0/2704139044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70102f60 0x7fae70103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 -- 172.21.15.96:0/2704139044 >> 172.21.15.96:0/2704139044 conn(0x7fae700fd340 msgr2=0x7fae700ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 -- 172.21.15.96:0/2704139044 shutdown_connections 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 -- 172.21.15.96:0/2704139044 wait complete. 2024-04-25T17:10:05.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 Processor -- start 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 -- start start 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae70102f60 0x7fae70198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.895+0000 7fae77f84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70103940 0x7fae70199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae77f84640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70104460 0x7fae701a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae77f84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae7010e530 con 0x7fae70103940 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae77f84640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae7010e3b0 con 0x7fae70104460 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae77f84640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fae7010e6b0 con 0x7fae70102f60 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70103940 0x7fae70199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70103940 0x7fae70199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58100/0 (socket says 172.21.15.96:58100) 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 -- 172.21.15.96:0/194515831 learned_addr learned my addr 172.21.15.96:0/194515831 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae75d20640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae70102f60 0x7fae70198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae76521640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70104460 0x7fae701a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 -- 172.21.15.96:0/194515831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae70102f60 msgr2=0x7fae70198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae70102f60 0x7fae70198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 -- 172.21.15.96:0/194515831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70104460 msgr2=0x7fae701a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70104460 0x7fae701a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 -- 172.21.15.96:0/194515831 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae701a0a10 con 0x7fae70103940 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae7551f640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70103940 0x7fae70199410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae6000d770 tx=0x7fae6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae66ffd640 1 -- 172.21.15.96:0/194515831 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60017070 con 0x7fae70103940 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae66ffd640 1 -- 172.21.15.96:0/194515831 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae600054d0 con 0x7fae70103940 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae701a0d00 con 0x7fae70103940 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae66ffd640 1 -- 172.21.15.96:0/194515831 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60004c30 con 0x7fae70103940 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.896+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701a11a0 con 0x7fae70103940 2024-04-25T17:10:05.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.897+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae70109910 con 0x7fae70103940 2024-04-25T17:10:05.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.899+0000 7fae66ffd640 1 -- 172.21.15.96:0/194515831 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fae60004dd0 con 0x7fae70103940 2024-04-25T17:10:05.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.899+0000 7fae66ffd640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae5c0afaf0 0x7fae5c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:05.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.899+0000 7fae75d20640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae5c0afaf0 0x7fae5c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:05.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.899+0000 7fae66ffd640 1 -- 172.21.15.96:0/194515831 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fae600deba0 con 0x7fae70103940 2024-04-25T17:10:05.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.900+0000 7fae75d20640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae5c0afaf0 0x7fae5c0b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae6c005c40 tx=0x7fae6c00a780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:05.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:05.903+0000 7fae66ffd640 1 -- 172.21.15.96:0/194515831 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae600a7780 con 0x7fae70103940 2024-04-25T17:10:05.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:05 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1022196129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:10:05.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:05 smithi097 ceph-mon[32602]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:10:05.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:05 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2097965791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:05.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:05 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1503469909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:10:06.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:05 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1022196129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:10:06.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:05 smithi171 ceph-mon[33189]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:10:06.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:05 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2097965791' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:06.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:05 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1503469909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:10:06.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.082+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae7002db50 con 0x7fae70103940 2024-04-25T17:10:06.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.082+0000 7fae66ffd640 1 -- 172.21.15.96:0/194515831 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7fae600a7080 con 0x7fae70103940 2024-04-25T17:10:06.081 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T17:10:06.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae5c0afaf0 msgr2=0x7fae5c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae5c0afaf0 0x7fae5c0b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae6c005c40 tx=0x7fae6c00a780 comp rx=0 tx=0).stop 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70103940 msgr2=0x7fae70199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70103940 0x7fae70199410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae6000d770 tx=0x7fae6000dc30 comp rx=0 tx=0).stop 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 shutdown_connections 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae70104460 0x7fae701a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae70103940 0x7fae70199410 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fae5c0afaf0 0x7fae5c0b2000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 --2- 172.21.15.96:0/194515831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fae70102f60 0x7fae70198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 >> 172.21.15.96:0/194515831 conn(0x7fae700fd340 msgr2=0x7fae70106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 shutdown_connections 2024-04-25T17:10:06.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.084+0000 7fae77f84640 1 -- 172.21.15.96:0/194515831 wait complete. 2024-04-25T17:10:06.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 -- 172.21.15.96:0/249951327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8104470 msgr2=0x7fc8e810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/249951327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8104470 0x7fc8e810abc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009ca0 tx=0x7fc8e002a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 -- 172.21.15.96:0/249951327 shutdown_connections 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/249951327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8104470 0x7fc8e810abc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/249951327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8e8103950 0x7fc8e8103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/249951327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e8102f70 0x7fc8e8103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 -- 172.21.15.96:0/249951327 >> 172.21.15.96:0/249951327 conn(0x7fc8e80fd370 msgr2=0x7fc8e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 -- 172.21.15.96:0/249951327 shutdown_connections 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.203+0000 7fc8eef4e640 1 -- 172.21.15.96:0/249951327 wait complete. 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 Processor -- start 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 -- start start 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8e8102f70 0x7fc8e819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8103950 0x7fc8e819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e8104470 0x7fc8e81a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810f640 con 0x7fc8e8103950 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810f4c0 con 0x7fc8e8104470 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8eef4e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810f7c0 con 0x7fc8e8102f70 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8103950 0x7fc8e819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8e7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8103950 0x7fc8e819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58122/0 (socket says 172.21.15.96:58122) 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.204+0000 7fc8e7fff640 1 -- 172.21.15.96:0/3086533534 learned_addr learned my addr 172.21.15.96:0/3086533534 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8eccea640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8e8102f70 0x7fc8e819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8ed4eb640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e8104470 0x7fc8e81a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e7fff640 1 -- 172.21.15.96:0/3086533534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8e8102f70 msgr2=0x7fc8e819a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e7fff640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8e8102f70 0x7fc8e819a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e7fff640 1 -- 172.21.15.96:0/3086533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e8104470 msgr2=0x7fc8e81a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e7fff640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e8104470 0x7fc8e81a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e7fff640 1 -- 172.21.15.96:0/3086533534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e81a1c70 con 0x7fc8e8103950 2024-04-25T17:10:06.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8eccea640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8e8102f70 0x7fc8e819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:06.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e7fff640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8103950 0x7fc8e819a670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400d950 tx=0x7fc8d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e5ffb640 1 -- 172.21.15.96:0/3086533534 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4017070 con 0x7fc8e8103950 2024-04-25T17:10:06.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e5ffb640 1 -- 172.21.15.96:0/3086533534 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d4009c30 con 0x7fc8e8103950 2024-04-25T17:10:06.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e81a1f60 con 0x7fc8e8103950 2024-04-25T17:10:06.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8e5ffb640 1 -- 172.21.15.96:0/3086533534 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d400f620 con 0x7fc8e8103950 2024-04-25T17:10:06.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.205+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e81a23d0 con 0x7fc8e8103950 2024-04-25T17:10:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.206+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e8109920 con 0x7fc8e8103950 2024-04-25T17:10:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.207+0000 7fc8e5ffb640 1 -- 172.21.15.96:0/3086533534 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc8d4010460 con 0x7fc8e8103950 2024-04-25T17:10:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.208+0000 7fc8e5ffb640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8d80afaf0 0x7fc8d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.208+0000 7fc8eccea640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8d80afaf0 0x7fc8d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.208+0000 7fc8e5ffb640 1 -- 172.21.15.96:0/3086533534 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fc8d40de8d0 con 0x7fc8e8103950 2024-04-25T17:10:06.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.208+0000 7fc8eccea640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8d80afaf0 0x7fc8d80b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005d00 tx=0x7fc8dc005ed0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.213+0000 7fc8e5ffb640 1 -- 172.21.15.96:0/3086533534 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8d40a74b0 con 0x7fc8e8103950 2024-04-25T17:10:06.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.428+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8e802dd70 con 0x7fc8e8103950 2024-04-25T17:10:06.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.429+0000 7fc8e5ffb640 1 -- 172.21.15.96:0/3086533534 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+11805 (secure 0 0 0) 0x7fc8d4009de0 con 0x7fc8e8103950 2024-04-25T17:10:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:10:06.428 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:08:14.735284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/825277463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":825277463},{"type":"v1","addr":"172.21.15.97:6837","nonce":825277463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:10:06.428 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 18 2024-04-25T17:10:06.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8d80afaf0 msgr2=0x7fc8d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8d80afaf0 0x7fc8d80b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005d00 tx=0x7fc8dc005ed0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8103950 msgr2=0x7fc8e819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8103950 0x7fc8e819a670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400d950 tx=0x7fc8d400de10 comp rx=0 tx=0).stop 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 shutdown_connections 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e8104470 0x7fc8e81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e8103950 0x7fc8e819a670 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8d80afaf0 0x7fc8d80b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 --2- 172.21.15.96:0/3086533534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8e8102f70 0x7fc8e819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 >> 172.21.15.96:0/3086533534 conn(0x7fc8e80fd370 msgr2=0x7fc8e8106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 shutdown_connections 2024-04-25T17:10:06.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.431+0000 7fc8eef4e640 1 -- 172.21.15.96:0/3086533534 wait complete. 2024-04-25T17:10:06.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3587890673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454108940 msgr2=0x7fe45410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3587890673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454108940 0x7fe45410ad60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009ca0 tx=0x7fe44c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3587890673 shutdown_connections 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3587890673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454108940 0x7fe45410ad60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3587890673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe454103c40 0x7fe454104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3587890673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe454102a00 0x7fe454103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3587890673 >> 172.21.15.96:0/3587890673 conn(0x7fe4540fdbf0 msgr2=0x7fe4541000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3587890673 shutdown_connections 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.551+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3587890673 wait complete. 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 Processor -- start 2024-04-25T17:10:06.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 -- start start 2024-04-25T17:10:06.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe454102a00 0x7fe4541990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454103c40 0x7fe454199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe454108940 0x7fe4541a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe45410e580 con 0x7fe454103c40 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe45410e400 con 0x7fe454108940 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe45a9d7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe45410e700 con 0x7fe454102a00 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454103c40 0x7fe454199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454103c40 0x7fe454199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58128/0 (socket says 172.21.15.96:58128) 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.552+0000 7fe4537fe640 1 -- 172.21.15.96:0/3615250826 learned_addr learned my addr 172.21.15.96:0/3615250826 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe458f74640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe454108940 0x7fe4541a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe453fff640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe454102a00 0x7fe4541990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4537fe640 1 -- 172.21.15.96:0/3615250826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe454102a00 msgr2=0x7fe4541990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4537fe640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe454102a00 0x7fe4541990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4537fe640 1 -- 172.21.15.96:0/3615250826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe454108940 msgr2=0x7fe4541a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4537fe640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe454108940 0x7fe4541a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4537fe640 1 -- 172.21.15.96:0/3615250826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4541a0c30 con 0x7fe454103c40 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe458f74640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe454108940 0x7fe4541a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe453fff640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe454102a00 0x7fe4541990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4537fe640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454103c40 0x7fe454199630 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe44800b4a0 tx=0x7fe44800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4517fa640 1 -- 172.21.15.96:0/3615250826 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448014070 con 0x7fe454103c40 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4517fa640 1 -- 172.21.15.96:0/3615250826 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe448010ac0 con 0x7fe454103c40 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4541a0f20 con 0x7fe454103c40 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe4517fa640 1 -- 172.21.15.96:0/3615250826 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448017c30 con 0x7fe454103c40 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.553+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4541a1410 con 0x7fe454103c40 2024-04-25T17:10:06.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.554+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe454107750 con 0x7fe454103c40 2024-04-25T17:10:06.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.556+0000 7fe4517fa640 1 -- 172.21.15.96:0/3615250826 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe448017070 con 0x7fe454103c40 2024-04-25T17:10:06.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.556+0000 7fe4517fa640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe43c0afaf0 0x7fe43c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.557+0000 7fe453fff640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe43c0afaf0 0x7fe43c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.557+0000 7fe4517fa640 1 -- 172.21.15.96:0/3615250826 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fe4480de3d0 con 0x7fe454103c40 2024-04-25T17:10:06.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.557+0000 7fe453fff640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe43c0afaf0 0x7fe43c0b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe454103600 tx=0x7fe44400a560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.563+0000 7fe4517fa640 1 -- 172.21.15.96:0/3615250826 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4480a6fb0 con 0x7fe454103c40 2024-04-25T17:10:06.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.742+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe45402dd20 con 0x7fe454103c40 2024-04-25T17:10:06.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.743+0000 7fe4517fa640 1 -- 172.21.15.96:0/3615250826 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22594 (secure 0 0 0) 0x7fe4480a68b0 con 0x7fe454103c40 2024-04-25T17:10:06.742 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:10:06.743 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":78,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:09:04.141423+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:10:06.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe43c0afaf0 msgr2=0x7fe43c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe43c0afaf0 0x7fe43c0b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe454103600 tx=0x7fe44400a560 comp rx=0 tx=0).stop 2024-04-25T17:10:06.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454103c40 msgr2=0x7fe454199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454103c40 0x7fe454199630 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe44800b4a0 tx=0x7fe44800b960 comp rx=0 tx=0).stop 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 shutdown_connections 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe454108940 0x7fe4541a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe454103c40 0x7fe454199630 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fe43c0afaf0 0x7fe43c0b2000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 --2- 172.21.15.96:0/3615250826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe454102a00 0x7fe4541990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 >> 172.21.15.96:0/3615250826 conn(0x7fe4540fdbf0 msgr2=0x7fe454104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 shutdown_connections 2024-04-25T17:10:06.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.745+0000 7fe45a9d7640 1 -- 172.21.15.96:0/3615250826 wait complete. 2024-04-25T17:10:06.755 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:06 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/410169574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:10:06.755 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:06 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/194515831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:10:06.755 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:06 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3086533534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:10:06.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T17:08:11.494613+0000', 'modified': '2024-04-25T17:08:14.735284+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/825277463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 825277463}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 825277463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6839/3298672156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3298672156}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3298672156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24521]} 2024-04-25T17:10:06.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T17:10:06.756 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 -- 172.21.15.96:0/618876682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854103c20 msgr2=0x7fc854104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 --2- 172.21.15.96:0/618876682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854103c20 0x7fc854104030 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc844009900 tx=0x7fc84402a410 comp rx=0 tx=0).stop 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 -- 172.21.15.96:0/618876682 shutdown_connections 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 --2- 172.21.15.96:0/618876682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc854108920 0x7fc85410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 --2- 172.21.15.96:0/618876682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854103c20 0x7fc854104030 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 --2- 172.21.15.96:0/618876682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8541029e0 0x7fc8541036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 -- 172.21.15.96:0/618876682 >> 172.21.15.96:0/618876682 conn(0x7fc8540fdbd0 msgr2=0x7fc854100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 -- 172.21.15.96:0/618876682 shutdown_connections 2024-04-25T17:10:06.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 -- 172.21.15.96:0/618876682 wait complete. 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.866+0000 7fc858a4c640 1 Processor -- start 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc858a4c640 1 -- start start 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc858a4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8541029e0 0x7fc85406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc858a4c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc854103c20 0x7fc85419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc858a4c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854108920 0x7fc8541a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc858a4c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410e510 con 0x7fc854108920 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc858a4c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410e390 con 0x7fc8541029e0 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc858a4c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc85410e690 con 0x7fc854103c20 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854108920 0x7fc8541a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854108920 0x7fc8541a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58146/0 (socket says 172.21.15.96:58146) 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 -- 172.21.15.96:0/2936733911 learned_addr learned my addr 172.21.15.96:0/2936733911 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc85259c640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8541029e0 0x7fc85406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc851d9b640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc854103c20 0x7fc85419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 -- 172.21.15.96:0/2936733911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc854103c20 msgr2=0x7fc85419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc854103c20 0x7fc85419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 -- 172.21.15.96:0/2936733911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8541029e0 msgr2=0x7fc85406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8541029e0 0x7fc85406a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 -- 172.21.15.96:0/2936733911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8541a2050 con 0x7fc854108920 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc851d9b640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc854103c20 0x7fc85419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc852d9d640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854108920 0x7fc8541a1830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00eaa0 tx=0x7fc84c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.867+0000 7fc84b7fe640 1 -- 172.21.15.96:0/2936733911 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00cdb0 con 0x7fc854108920 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.868+0000 7fc84b7fe640 1 -- 172.21.15.96:0/2936733911 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc84c005020 con 0x7fc854108920 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.868+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8541a2340 con 0x7fc854108920 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.868+0000 7fc84b7fe640 1 -- 172.21.15.96:0/2936733911 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0116d0 con 0x7fc854108920 2024-04-25T17:10:06.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.868+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a2890 con 0x7fc854108920 2024-04-25T17:10:06.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 Processor -- start 2024-04-25T17:10:06.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc84b7fe640 1 -- 172.21.15.96:0/2936733911 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fc84c0051d0 con 0x7fc854108920 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 -- start start 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddf2270 0x5568de193370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 0x5568de19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568de0f8740 con 0x5568dddf2270 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568de0943a0 con 0x5568dddc93b0 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc85f9db740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5568de0f4700 con 0x5568dddd1590 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.870+0000 7fc851d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddf2270 0x5568de193370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddf2270 0x5568de193370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58160/0 (socket says 172.21.15.96:58160) 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 -- 172.21.15.96:0/140812634 learned_addr learned my addr 172.21.15.96:0/140812634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc85259c640 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 -- 172.21.15.96:0/140812634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 msgr2=0x5568de19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 0x5568de19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc84b7fe640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8400afaf0 0x7fc8400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 -- 172.21.15.96:0/140812634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 msgr2=0x5568de1938f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1938f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 -- 172.21.15.96:0/140812634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568de0b0750 con 0x5568dddf2270 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc85259c640 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1938f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc852d9d640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8400afaf0 0x7fc8400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc84b7fe640 1 -- 172.21.15.96:0/2936733911 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fc84c0ded40 con 0x7fc854108920 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc851d9b640 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddf2270 0x5568de193370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc844005940 tx=0x7fc844032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73085a3886cb8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc848ff9640 1 -- 172.21.15.96:0/140812634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844040070 con 0x5568dddf2270 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc848ff9640 1 -- 172.21.15.96:0/140812634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8440317a0 con 0x5568dddf2270 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.871+0000 7fc848ff9640 1 -- 172.21.15.96:0/140812634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844035050 con 0x5568dddf2270 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 -- 172.21.15.96:0/140812634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddf2270 msgr2=0x5568de193370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddf2270 0x5568de193370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc844005940 tx=0x7fc844032c60 comp rx=0 tx=0).stop 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc852d9d640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8400afaf0 0x7fc8400b2000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0a6f20 tx=0x7fc84c06d740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 -- 172.21.15.96:0/140812634 shutdown_connections 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 0x5568de19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1938f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 --2- 172.21.15.96:0/140812634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddf2270 0x5568de193370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 -- 172.21.15.96:0/140812634 >> 172.21.15.96:0/140812634 conn(0x5568ddde63c0 msgr2=0x5568de18ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 -- 172.21.15.96:0/140812634 shutdown_connections 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.872+0000 7fc85f9db740 1 -- 172.21.15.96:0/140812634 wait complete. 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 -- start start 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 Processor -- start 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de19d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddd1590 0x5568de19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddf2270 0x5568de1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568de19b1e0 con 0x5568dddd1590 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568de07ffe0 con 0x5568dddc93b0 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5568de19b360 con 0x5568dddf2270 2024-04-25T17:10:06.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddd1590 0x5568de19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85f9db740 10 client.0 _ll_register_callbacks cb 0x5568de096150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddd1590 0x5568de19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58170/0 (socket says 172.21.15.96:58170) 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 -- 172.21.15.96:0/2888525939 learned_addr learned my addr 172.21.15.96:0/2888525939 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc851d9b640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc851d9b640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddf2270 0x5568de1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 -- 172.21.15.96:0/2888525939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddf2270 msgr2=0x5568de1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddf2270 0x5568de1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 -- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 msgr2=0x5568de19d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de19d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 -- 172.21.15.96:0/2888525939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568dde0a2d0 con 0x5568dddd1590 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc851d9b640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de19d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc85259c640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddd1590 0x5568de19d7d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00ba50 tx=0x7fc83c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5518b47a40e6d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.873+0000 7fc839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc839ffb640 1 -- 172.21.15.96:0/2888525939 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c011640 con 0x5568dddd1590 2024-04-25T17:10:06.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc839ffb640 1 -- 172.21.15.96:0/2888525939 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fc83c0117e0 con 0x5568dddd1590 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8440317a0 con 0x5568dddd1590 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc839ffb640 1 -- 172.21.15.96:0/2888525939 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc83c0119e0 con 0x5568dddd1590 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 10 client.15234 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5568dded46b0 con 0x5568dddd1590 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc839ffb640 1 -- 172.21.15.96:0/2888525939 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8440317a0 con 0x5568dddd1590 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc839ffb640 1 -- 172.21.15.96:0/2888525939 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc83c010c70 con 0x5568dddd1590 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 10 client.15234 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 20 client.15234 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddb0e40 0x5568de1ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 4 client.15234 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568de106350 con 0x5568dddb0e40 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.874+0000 7fc851d9b640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddb0e40 0x5568de1ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.875+0000 7fc839ffb640 10 client.15234 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:06.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.875+0000 7fc851d9b640 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddb0e40 0x5568de1ab0a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dffb300248309e76 server_cookie=5d1fa1801acad6c4 in_seq=0 out_seq=0 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.875+0000 7fc839ffb640 1 -- 172.21.15.96:0/2888525939 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568de106350 con 0x5568dddb0e40 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.875+0000 7fc839ffb640 10 client.15234 handle_command_reply: tid=0 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 client.15234 shutdown 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddd1590 msgr2=0x5568de19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddd1590 0x5568de19d7d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00ba50 tx=0x7fc83c00be50 comp rx=0 tx=0).stop 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 shutdown_connections 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddb0e40 0x5568de1ab0a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddf2270 0x5568de1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddd1590 0x5568de19d7d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 --2- 172.21.15.96:0/2888525939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de19d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc839ffb640 0 client.15234 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 >> 172.21.15.96:0/2888525939 conn(0x5568ddde63c0 msgr2=0x5568de18dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 shutdown_connections 2024-04-25T17:10:06.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 1 -- 172.21.15.96:0/2888525939 wait complete. 2024-04-25T17:10:06.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.876+0000 7fc85f9db740 20 client.15234 trim_cache size 0 max 16384 2024-04-25T17:10:06.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 Processor -- start 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 -- start start 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddb0e40 0x5568de1a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 0x5568de19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568de1a93e0 con 0x5568dddb0e40 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568de1a9ab0 con 0x5568dddc93b0 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc85f9db740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5568de1aa180 con 0x5568dddd1590 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc851d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddb0e40 0x5568de1a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc851d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddb0e40 0x5568de1a8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58186/0 (socket says 172.21.15.96:58186) 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc851d9b640 1 -- 172.21.15.96:0/1111470193 learned_addr learned my addr 172.21.15.96:0/1111470193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc851d9b640 1 -- 172.21.15.96:0/1111470193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 msgr2=0x5568de19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.894+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1111470193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 0x5568de19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc851d9b640 1 -- 172.21.15.96:0/1111470193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 msgr2=0x5568de1a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1111470193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc851d9b640 1 -- 172.21.15.96:0/1111470193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568de198140 con 0x5568dddb0e40 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1111470193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddb0e40 0x5568de1a8a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc844006010 tx=0x7fc84402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf6183f178ef506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc83bfff640 1 -- 172.21.15.96:0/1111470193 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844057070 con 0x5568dddb0e40 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc83bfff640 1 -- 172.21.15.96:0/1111470193 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc844050070 con 0x5568dddb0e40 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc83bfff640 1 -- 172.21.15.96:0/1111470193 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844035040 con 0x5568dddb0e40 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc85f9db740 1 -- 172.21.15.96:0/1111470193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddb0e40 msgr2=0x5568de1a8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1111470193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddb0e40 0x5568de1a8a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc844006010 tx=0x7fc84402a410 comp rx=0 tx=0).stop 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc85f9db740 1 -- 172.21.15.96:0/1111470193 shutdown_connections 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1111470193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddd1590 0x5568de19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1111470193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddc93b0 0x5568de1a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1111470193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddb0e40 0x5568de1a8a00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.895+0000 7fc85f9db740 1 -- 172.21.15.96:0/1111470193 >> 172.21.15.96:0/1111470193 conn(0x5568ddde63c0 msgr2=0x5568de1969c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 -- 172.21.15.96:0/1111470193 shutdown_connections 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 -- 172.21.15.96:0/1111470193 wait complete. 2024-04-25T17:10:06.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 -- start start 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 Processor -- start 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddb0e40 0x5568de19f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddc93b0 0x5568de19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddd1590 0x5568de211f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5568de1aaf20 con 0x5568dddc93b0 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568de1aa850 con 0x5568dddd1590 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5568de1ab5f0 con 0x5568dddb0e40 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85f9db740 10 client.0 _ll_register_callbacks cb 0x7fc83c004200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddc93b0 0x5568de19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddc93b0 0x5568de19f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58192/0 (socket says 172.21.15.96:58192) 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 -- 172.21.15.96:0/1628455826 learned_addr learned my addr 172.21.15.96:0/1628455826 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddb0e40 0x5568de19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 -- 172.21.15.96:0/1628455826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddb0e40 msgr2=0x5568de19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddd1590 0x5568de211f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddb0e40 0x5568de19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 -- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddd1590 msgr2=0x5568de211f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddd1590 0x5568de211f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc85259c640 1 -- 172.21.15.96:0/1628455826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568de1a48a0 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.896+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddd1590 0x5568de211f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85259c640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddc93b0 0x5568de19f8d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc83c01b680 tx=0x7fc83c005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d47ffaf0ea87ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc839ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc839ffb640 1 -- 172.21.15.96:0/1628455826 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c01bad0 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc839ffb640 1 -- 172.21.15.96:0/1628455826 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fc83c011440 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc844050070 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc839ffb640 1 -- 172.21.15.96:0/1628455826 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc83c011640 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 10 client.15240 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x5568de2134a0 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc839ffb640 1 -- 172.21.15.96:0/1628455826 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc844050070 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc839ffb640 1 -- 172.21.15.96:0/1628455826 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fc83c026460 con 0x5568dddc93b0 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 10 client.15240 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:06.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 20 client.15240 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddf2270 0x5568de216ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 4 client.15240 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.897+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "status"}) -- 0x5568de1001c0 con 0x5568dddf2270 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddf2270 0x5568de216ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc851d9b640 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddf2270 0x5568de216ee0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92202b6eb3c3f95 server_cookie=1afa6fb973909572 in_seq=0 out_seq=0 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc839ffb640 10 client.15240 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc839ffb640 1 -- 172.21.15.96:0/1628455826 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5568de1001c0 con 0x5568dddf2270 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc839ffb640 10 client.15240 handle_command_reply: tid=0 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 client.15240 shutdown 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddc93b0 msgr2=0x5568de19f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddc93b0 0x5568de19f8d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc83c01b680 tx=0x7fc83c005690 comp rx=0 tx=0).stop 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 shutdown_connections 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5568dddf2270 0x5568de216ee0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568dddd1590 0x5568de211f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc839ffb640 0 client.15240 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5568dddc93b0 0x5568de19f8d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.898+0000 7fc85f9db740 1 --2- 172.21.15.96:0/1628455826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568dddb0e40 0x5568de19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 >> 172.21.15.96:0/1628455826 conn(0x5568ddde63c0 msgr2=0x5568de18e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 shutdown_connections 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc85f9db740 1 -- 172.21.15.96:0/1628455826 wait complete. 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc85f9db740 20 client.15240 trim_cache size 0 max 16384 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "1be09d02-0325-11ef-bc93-c7b262605968", 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24521, 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T17:10:06.904 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T17:10:06.905 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T17:10:06.905 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 114.759720172, 2024-04-25T17:10:06.905 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 18, 2024-04-25T17:10:06.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 78, 2024-04-25T17:10:06.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 76, 2024-04-25T17:10:06.905 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 121.80098083 2024-04-25T17:10:06.905 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8400afaf0 msgr2=0x7fc8400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8400afaf0 0x7fc8400b2000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0a6f20 tx=0x7fc84c06d740 comp rx=0 tx=0).stop 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854108920 msgr2=0x7fc8541a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854108920 0x7fc8541a1830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00eaa0 tx=0x7fc84c00ef60 comp rx=0 tx=0).stop 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 shutdown_connections 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc854108920 0x7fc8541a1830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fc8400afaf0 0x7fc8400b2000 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc854103c20 0x7fc85419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 --2- 172.21.15.96:0/2936733911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8541029e0 0x7fc85406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 >> 172.21.15.96:0/2936733911 conn(0x7fc8540fdbd0 msgr2=0x7fc854104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 shutdown_connections 2024-04-25T17:10:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:06.899+0000 7fc858a4c640 1 -- 172.21.15.96:0/2936733911 wait complete. 2024-04-25T17:10:06.909 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T17:10:06.919 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T17:10:06.919 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T17:10:06.919 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T17:10:06.919 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T17:10:06.920 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T17:10:06.920 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:10:06.920 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T17:10:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:06 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/410169574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:10:06.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:06 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/194515831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:10:06.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:06 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3086533534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 -- 172.21.15.96:0/1651814629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 msgr2=0x7f709410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 --2- 172.21.15.96:0/1651814629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f709410be90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f708c009ca0 tx=0x7f708c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 -- 172.21.15.96:0/1651814629 shutdown_connections 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 --2- 172.21.15.96:0/1651814629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f709410be90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 --2- 172.21.15.96:0/1651814629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7094104d70 0x7f7094105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 --2- 172.21.15.96:0/1651814629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7094103b30 0x7f7094104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 -- 172.21.15.96:0/1651814629 >> 172.21.15.96:0/1651814629 conn(0x7f70940fed40 msgr2=0x7f70941011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 -- 172.21.15.96:0/1651814629 shutdown_connections 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.028+0000 7f709c2a7640 1 -- 172.21.15.96:0/1651814629 wait complete. 2024-04-25T17:10:07.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 Processor -- start 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 -- start start 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7094103b30 0x7f709419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7094104d70 0x7f709419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f70941a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f709410f660 con 0x7f7094109a70 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f709410f4e0 con 0x7f7094103b30 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709c2a7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f709410f7e0 con 0x7f7094104d70 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a844640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f70941a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a844640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f70941a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58218/0 (socket says 172.21.15.96:58218) 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a844640 1 -- 172.21.15.96:0/23698588 learned_addr learned my addr 172.21.15.96:0/23698588 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a043640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7094103b30 0x7f709419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a844640 1 -- 172.21.15.96:0/23698588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7094104d70 msgr2=0x7f709419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a844640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7094104d70 0x7f709419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a844640 1 -- 172.21.15.96:0/23698588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7094103b30 msgr2=0x7f709419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.029+0000 7f709a844640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7094103b30 0x7f709419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.030+0000 7f709a844640 1 -- 172.21.15.96:0/23698588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70941a1c60 con 0x7f7094109a70 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.030+0000 7f709a844640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f70941a16a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f708c0097b0 tx=0x7f708c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.030+0000 7f708b7fe640 1 -- 172.21.15.96:0/23698588 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c003bf0 con 0x7f7094109a70 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.030+0000 7f708b7fe640 1 -- 172.21.15.96:0/23698588 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708c003d90 con 0x7f7094109a70 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.030+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70941a1ef0 con 0x7f7094109a70 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.030+0000 7f708b7fe640 1 -- 172.21.15.96:0/23698588 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c02fc30 con 0x7f7094109a70 2024-04-25T17:10:07.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.030+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70941a23b0 con 0x7f7094109a70 2024-04-25T17:10:07.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:06 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/410169574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:10:07.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:06 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/194515831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:10:07.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:06 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3086533534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:10:07.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.031+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7094108880 con 0x7f7094109a70 2024-04-25T17:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.033+0000 7f708b7fe640 1 -- 172.21.15.96:0/23698588 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f708c004b40 con 0x7f7094109a70 2024-04-25T17:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.033+0000 7f708b7fe640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70800afae0 0x7f70800b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.034+0000 7f709a043640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70800afae0 0x7f70800b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.034+0000 7f708b7fe640 1 -- 172.21.15.96:0/23698588 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f708c100550 con 0x7f7094109a70 2024-04-25T17:10:07.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.034+0000 7f709a043640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70800afae0 0x7f70800b1ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7094104730 tx=0x7f7090005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:07.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.037+0000 7f708b7fe640 1 -- 172.21.15.96:0/23698588 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f708c0c90f0 con 0x7f7094109a70 2024-04-25T17:10:07.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.263+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f709402dce0 con 0x7f7094109a70 2024-04-25T17:10:07.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.563+0000 7f708b7fe640 1 -- 172.21.15.96:0/23698588 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f708c0c89f0 con 0x7f7094109a70 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.565+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70800afae0 msgr2=0x7f70800b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70800afae0 0x7f70800b1ff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7094104730 tx=0x7f7090005f90 comp rx=0 tx=0).stop 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 msgr2=0x7f70941a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f70941a16a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f708c0097b0 tx=0x7f708c031840 comp rx=0 tx=0).stop 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 shutdown_connections 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7094109a70 0x7f70941a16a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f70800afae0 0x7f70800b1ff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7094104d70 0x7f709419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 --2- 172.21.15.96:0/23698588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7094103b30 0x7f709419a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 >> 172.21.15.96:0/23698588 conn(0x7f70940fed40 msgr2=0x7f7094105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 shutdown_connections 2024-04-25T17:10:07.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:07.566+0000 7f709c2a7640 1 -- 172.21.15.96:0/23698588 wait complete. 2024-04-25T17:10:07.582 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T17:10:07.604 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T17:10:07.604 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T17:10:07.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:07.604 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T17:10:07.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:07.620 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-04-25T17:10:07.685 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-04-25T17:10:07.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:10:07.687 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T17:10:07.687 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T17:10:07.687 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:07.687 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T17:10:07.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:07 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3615250826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:07.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:07 smithi096 ceph-mon[29131]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:07.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:07 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/23698588' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T17:10:07.939 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: ignoring --setuser ceph since I am not root 2024-04-25T17:10:07.939 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: ignoring --setgroup ceph since I am not root 2024-04-25T17:10:07.939 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:07.776+0000 7f475c85a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:10:07.954 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3615250826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:07.954 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-mon[32602]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:07.954 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/23698588' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T17:10:07.955 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: ignoring --setuser ceph since I am not root 2024-04-25T17:10:07.955 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: ignoring --setgroup ceph since I am not root 2024-04-25T17:10:07.955 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-mgr[33993]: -- 172.21.15.97:0/3096899864 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578adbdc5a0 con 0x5578b2966900 2024-04-25T17:10:07.955 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:07.767+0000 7fcda5670200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:10:07.955 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:07.954+0000 7fcda5670200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:10:07.959 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3615250826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:10:07.959 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-mon[33189]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:07.960 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/23698588' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T17:10:07.960 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: ignoring --setuser ceph since I am not root 2024-04-25T17:10:07.960 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: ignoring --setgroup ceph since I am not root 2024-04-25T17:10:07.960 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-mgr[35073]: -- 172.21.15.171:0/1081176024 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600a0a625a0 con 0x5600a57ec000 2024-04-25T17:10:07.960 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:07.770+0000 7fae9ecdd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T17:10:07.960 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:07.959+0000 7fae9ecdd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:10:07.987 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:10:08.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:08.058+0000 7fcda5670200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:10:08.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:07.992+0000 7f475c85a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:10:08.247 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:08 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:08.101+0000 7f475c85a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:10:08.279 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:08.062+0000 7fae9ecdd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T17:10:08.514 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:08 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:08.426+0000 7f475c85a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:10:08.514 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:08 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:08.515+0000 7f475c85a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:10:08.733 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:08.367+0000 7fcda5670200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:10:08.734 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:08.454+0000 7fcda5670200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:10:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:08 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/23698588' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T17:10:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:08 smithi097 ceph-mon[32602]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T17:10:08.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:08 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/23698588' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T17:10:08.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:08 smithi171 ceph-mon[33189]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T17:10:08.780 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:08.374+0000 7fae9ecdd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T17:10:08.780 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:08.461+0000 7fae9ecdd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T17:10:08.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:08 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/23698588' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T17:10:08.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:08 smithi096 ceph-mon[29131]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T17:10:09.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 -- 172.21.15.96:0/1144900349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 msgr2=0x7f43a410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 --2- 172.21.15.96:0/1144900349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 0x7f43a410ff70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f439c00b0d0 tx=0x7f439c00b3e0 comp rx=0 tx=0).stop 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 -- 172.21.15.96:0/1144900349 shutdown_connections 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 --2- 172.21.15.96:0/1144900349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 0x7f43a410ff70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 --2- 172.21.15.96:0/1144900349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43a4103390 0x7f43a41037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 --2- 172.21.15.96:0/1144900349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4107cc0 0x7f43a41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 -- 172.21.15.96:0/1144900349 >> 172.21.15.96:0/1144900349 conn(0x7f43a40fec90 msgr2=0x7f43a41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 -- 172.21.15.96:0/1144900349 shutdown_connections 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.797+0000 7f43aa343700 1 -- 172.21.15.96:0/1144900349 wait complete. 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 Processor -- start 2024-04-25T17:10:09.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 -- start start 2024-04-25T17:10:09.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4103390 0x7f43a419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:09.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43a4107cc0 0x7f43a419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:09.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 0x7f43a4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:09.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43a4113260 con 0x7f43a4108610 2024-04-25T17:10:09.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41130e0 con 0x7f43a4103390 2024-04-25T17:10:09.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43aa343700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f43a41133e0 con 0x7f43a4107cc0 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43a88e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 0x7f43a4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4103390 0x7f43a419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43a37fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43a4107cc0 0x7f43a419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43a3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4103390 0x7f43a419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:39716/0 (socket says 172.21.15.96:39716) 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43a37fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43a4107cc0 0x7f43a419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:56692/0 (socket says 172.21.15.96:56692) 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43a88e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 0x7f43a4199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55544/0 (socket says 172.21.15.96:55544) 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.798+0000 7f43a3fff700 1 -- 172.21.15.96:0/2552324664 learned_addr learned my addr 172.21.15.96:0/2552324664 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a3fff700 1 -- 172.21.15.96:0/2552324664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43a4107cc0 msgr2=0x7f43a419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a3fff700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43a4107cc0 0x7f43a419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a3fff700 1 -- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 msgr2=0x7f43a4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a3fff700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 0x7f43a4199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a3fff700 1 -- 172.21.15.96:0/2552324664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a419a110 con 0x7f43a4103390 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a3fff700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4103390 0x7f43a419f840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4394010bd0 tx=0x7f4394010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a17fa700 1 -- 172.21.15.96:0/2552324664 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439400ca90 con 0x7f43a4103390 2024-04-25T17:10:09.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a419a400 con 0x7f43a4103390 2024-04-25T17:10:09.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a17fa700 1 -- 172.21.15.96:0/2552324664 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4394004ce0 con 0x7f43a4103390 2024-04-25T17:10:09.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43a17fa700 1 -- 172.21.15.96:0/2552324664 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394005650 con 0x7f43a4103390 2024-04-25T17:10:09.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.799+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a406b4e0 con 0x7f43a4103390 2024-04-25T17:10:09.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.800+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a4104840 con 0x7f43a4103390 2024-04-25T17:10:09.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.802+0000 7f43a17fa700 1 -- 172.21.15.96:0/2552324664 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f439401f020 con 0x7f43a4103390 2024-04-25T17:10:09.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.802+0000 7f43a17fa700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 0x7f438c0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:09.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.802+0000 7f43a37fe700 1 -- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 msgr2=0x7f438c0b20e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:09.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.802+0000 7f43a17fa700 1 -- 172.21.15.96:0/2552324664 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f43940e0470 con 0x7f43a4103390 2024-04-25T17:10:09.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.802+0000 7f43a37fe700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 0x7f438c0b20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:10:09.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:09.807+0000 7f43a17fa700 1 -- 172.21.15.96:0/2552324664 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4394053ff0 con 0x7f43a4103390 2024-04-25T17:10:09.938 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:09.731+0000 7f475c85a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:10:09.938 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:09.914+0000 7f475c85a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:10:09.981 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:09.637+0000 7fcda5670200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:10:09.982 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:09.810+0000 7fcda5670200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:10:10.001 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:09.651+0000 7fae9ecdd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T17:10:10.001 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:09.826+0000 7fae9ecdd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T17:10:10.001 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:10.000+0000 7fae9ecdd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:10:10.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.003+0000 7f43a37fe700 1 -- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 msgr2=0x7f438c0b20e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:10.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.003+0000 7f43a37fe700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 0x7f438c0b20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T17:10:10.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.054+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f43a419af90 con 0x7f43a4103390 2024-04-25T17:10:10.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.055+0000 7f43a17fa700 1 -- 172.21.15.96:0/2552324664 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f43940a9050 con 0x7f43a4103390 2024-04-25T17:10:10.054 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T17:10:10.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 msgr2=0x7f438c0b20e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:10:10.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 0x7f438c0b20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:10.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4103390 msgr2=0x7f43a419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4103390 0x7f43a419f840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4394010bd0 tx=0x7f4394010ee0 comp rx=0 tx=0).stop 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 shutdown_connections 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f438c0afbd0 0x7f438c0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43a4108610 0x7f43a4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43a4103390 0x7f43a419f840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 --2- 172.21.15.96:0/2552324664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43a4107cc0 0x7f43a419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 >> 172.21.15.96:0/2552324664 conn(0x7f43a40fec90 msgr2=0x7f43a4107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 shutdown_connections 2024-04-25T17:10:10.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10.057+0000 7f43aa343700 1 -- 172.21.15.96:0/2552324664 wait complete. 2024-04-25T17:10:10.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:10 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2552324664' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T17:10:10.234 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:09.981+0000 7fcda5670200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:10:10.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:10 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2552324664' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T17:10:10.370 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:10.091+0000 7f475c85a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T17:10:10.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:10 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2552324664' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T17:10:10.572 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:10.277+0000 7fcda5670200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:10:10.572 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:10.370+0000 7fcda5670200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:10:10.592 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:10.293+0000 7fae9ecdd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:10:10.593 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:10.387+0000 7fae9ecdd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:10:10.654 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:10.392+0000 7f475c85a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T17:10:10.654 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:10.486+0000 7f475c85a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T17:10:10.683 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T17:10:10.683 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T17:10:10.683 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-04-25T17:10:10.683 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T17:10:10.683 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:10.684 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T17:10:10.684 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T17:10:10.684 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T17:10:10.684 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T17:10:10.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:10.684 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-04-25T17:10:10.699 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T17:10:10.699 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T17:10:10.699 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-04-25T17:10:10.700 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T17:10:10.701 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42119sec preferred_lft 42119sec 2024-04-25T17:10:10.701 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ddd4:c54b:dd9f:768b/64 scope link noprefixroute 2024-04-25T17:10:10.701 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T17:10:10.701 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T17:10:10.701 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T17:10:10.701 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T17:10:10.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-04-25T17:10:10.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-04-25T17:10:10.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-04-25T17:10:10.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T17:10:10.701 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T17:10:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T17:10:10.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:10.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:10.907 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T17:10:10.917 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:10.697+0000 7f475c85a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:10:10.917 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:10.904+0000 7f475c85a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:10:10.932 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-04-25T17:10:10.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:10.934 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-04-25T17:10:10.983 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:10.572+0000 7fcda5670200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:10:10.983 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:10.767+0000 7fcda5670200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:10:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-04-25T17:10:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T17:10:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T17:10:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T17:10:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T17:10:11.000 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T17:10:11.000 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T17:10:11.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T17:10:11.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T17:10:11.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T17:10:11.000 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T17:10:11.030 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:10.592+0000 7fae9ecdd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T17:10:11.030 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:10.789+0000 7fae9ecdd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T17:10:11.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T17:10:11.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:11.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:11.273 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T17:10:11.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:11.288 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-04-25T17:10:11.342 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T17:10:11.342 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T17:10:11.343 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T17:10:11.343 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T17:10:11.343 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T17:10:11.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T17:10:11.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:11.463 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T17:10:11.464 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T17:10:11.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T17:10:11.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:11.595 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T17:10:11.595 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T17:10:11.595 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-04-25T17:10:11.595 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T17:10:11.595 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T17:10:11.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T17:10:11.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:11.671 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T17:10:11.671 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:11.685 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T17:10:11.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T17:10:11.686 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:11.733 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:11.371+0000 7fcda5670200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:10:11.740 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T17:10:11.740 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T17:10:11.740 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/87e8df53-ac6d-498d-95ee-67a03515551e 2024-04-25T17:10:11.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:11.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/87e8df53-ac6d-498d-95ee-67a03515551e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T17:10:11.779 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:11.403+0000 7fae9ecdd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:10:11.801 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:11 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:11.544+0000 7f475c85a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T17:10:11.842 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T17:10:11.842 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-25T17:10:11.842 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-04-25T17:10:11.963 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T17:10:11.963 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-25T17:10:11.963 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-04-25T17:10:11.963 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.96:3300,172.21.15.97:3300,172.21.15.171:3300" 2024-04-25T17:10:11.963 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.96:3300,172.21.15.97:3300,172.21.15.171:3300:/volumes/qa/sv_1/87e8df53-ac6d-498d-95ee-67a03515551e 2024-04-25T17:10:11.963 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=1be09d02-0325-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T17:10:11.964 INFO:teuthology.orchestra.run.smithi096.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T17:10:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: You just mounted a file system that supports labels which does not 2024-04-25T17:10:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T17:10:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T17:10:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T17:10:11.966 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T17:10:11.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:10:11.966 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:12.032 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T17:10:12.106 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:11.802+0000 7fcda5670200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:10:12.106 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:11.900+0000 7fcda5670200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:10:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T17:10:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:12.126 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:12.024+0000 7f475c85a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:10:12.132 INFO:teuthology.orchestra.run.smithi096.stdout:client24911 2024-04-25T17:10:12.132 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:10:12.132 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T17:10:12.139 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:11.834+0000 7fae9ecdd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T17:10:12.139 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:11.932+0000 7fae9ecdd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:10:12.148 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T17:10:12.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T17:10:12.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:12.245 INFO:teuthology.orchestra.run.smithi096.stdout:client24911 2024-04-25T17:10:12.245 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T17:10:12.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T17:10:12.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:client24911 2024-04-25T17:10:12.302 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1be09d02-0325-11ef-bc93-c7b262605968.client24911/status') 2024-04-25T17:10:12.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T17:10:12.359 INFO:teuthology.orchestra.run.smithi096.stdout:instance: client.24911 (3)172.21.15.96:0/1027904856 2024-04-25T17:10:12.359 INFO:teuthology.orchestra.run.smithi096.stdout:blocklisted: false 2024-04-25T17:10:12.359 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2024-04-25T17:10:12.359 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2024-04-25T17:10:12.359 INFO:teuthology.orchestra.run.smithi096.stderr:71 bytes copied, 2.3004e-05 s, 3.1 MB/s 2024-04-25T17:10:12.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T17:10:12.366 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T17:10:12.378 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T17:10:12.398 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T17:10:12.398 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:10:12.399 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-04-25T17:10:12.420 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:12.127+0000 7f475c85a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T17:10:12.420 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:12.352+0000 7f475c85a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:10:12.483 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:12.106+0000 7fcda5670200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:10:12.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 -- 172.21.15.96:0/472918781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4101790 msgr2=0x7fbeb4101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/472918781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4101790 0x7fbeb4101c00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009900 tx=0x7fbe9c02a410 comp rx=0 tx=0).stop 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 -- 172.21.15.96:0/472918781 shutdown_connections 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/472918781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb4102290 0x7fbeb410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/472918781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4101790 0x7fbeb4101c00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/472918781 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeb41050e0 0x7fbeb41054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 -- 172.21.15.96:0/472918781 >> 172.21.15.96:0/472918781 conn(0x7fbeb40fd390 msgr2=0x7fbeb40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.514+0000 7fbeb8e13640 1 -- 172.21.15.96:0/472918781 shutdown_connections 2024-04-25T17:10:12.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.515+0000 7fbeb8e13640 1 -- 172.21.15.96:0/472918781 wait complete. 2024-04-25T17:10:12.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.515+0000 7fbeb8e13640 1 Processor -- start 2024-04-25T17:10:12.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.515+0000 7fbeb8e13640 1 -- start start 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb8e13640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeb4101790 0x7fbeb419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb8e13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4102290 0x7fbeb419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb8e13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb41050e0 0x7fbeb41a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb8e13640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb410f6c0 con 0x7fbeb4102290 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb8e13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb410f540 con 0x7fbeb41050e0 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb8e13640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb410f840 con 0x7fbeb4101790 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4102290 0x7fbeb419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4102290 0x7fbeb419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55562/0 (socket says 172.21.15.96:55562) 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 -- 172.21.15.96:0/2815593405 learned_addr learned my addr 172.21.15.96:0/2815593405 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb2d9d640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb41050e0 0x7fbeb41a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb259c640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeb4101790 0x7fbeb419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 -- 172.21.15.96:0/2815593405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeb4101790 msgr2=0x7fbeb419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeb4101790 0x7fbeb419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 -- 172.21.15.96:0/2815593405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb41050e0 msgr2=0x7fbeb41a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb41050e0 0x7fbeb41a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 -- 172.21.15.96:0/2815593405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb41a1d70 con 0x7fbeb4102290 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeabfff640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4102290 0x7fbeb419a770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c031440 tx=0x7fbe9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeab7fe640 1 -- 172.21.15.96:0/2815593405 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c03f070 con 0x7fbeb4102290 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeab7fe640 1 -- 172.21.15.96:0/2815593405 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9c031c50 con 0x7fbeb4102290 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41a2000 con 0x7fbeb4102290 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.516+0000 7fbeab7fe640 1 -- 172.21.15.96:0/2815593405 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c035030 con 0x7fbeb4102290 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.517+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb41a24f0 con 0x7fbeb4102290 2024-04-25T17:10:12.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.518+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb4108a70 con 0x7fbeb4102290 2024-04-25T17:10:12.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.519+0000 7fbeab7fe640 1 -- 172.21.15.96:0/2815593405 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7fbe9c042070 con 0x7fbeb4102290 2024-04-25T17:10:12.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.520+0000 7fbeab7fe640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbea00afb30 0x7fbea00b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.520+0000 7fbeb259c640 1 -- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbea00afb30 msgr2=0x7fbea00b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:12.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.520+0000 7fbeab7fe640 1 -- 172.21.15.96:0/2815593405 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7fbe9c0ffb30 con 0x7fbeb4102290 2024-04-25T17:10:12.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.520+0000 7fbeb259c640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbea00afb30 0x7fbea00b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:10:12.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.524+0000 7fbeab7fe640 1 -- 172.21.15.96:0/2815593405 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe9c0c80c0 con 0x7fbeb4102290 2024-04-25T17:10:12.529 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:12.138+0000 7fae9ecdd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T17:10:12.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.710+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fbeb402dbf0 con 0x7fbeb4102290 2024-04-25T17:10:12.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.714+0000 7fbeab7fe640 1 -- 172.21.15.96:0/2815593405 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7fbe9c0cc3c0 con 0x7fbeb4102290 2024-04-25T17:10:12.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbea00afb30 msgr2=0x7fbea00b2040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbea00afb30 0x7fbea00b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4102290 msgr2=0x7fbeb419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4102290 0x7fbeb419a770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c031440 tx=0x7fbe9c003c10 comp rx=0 tx=0).stop 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 shutdown_connections 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbeb41050e0 0x7fbeb41a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeb4102290 0x7fbeb419a770 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7fbea00afb30 0x7fbea00b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 --2- 172.21.15.96:0/2815593405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbeb4101790 0x7fbeb419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 >> 172.21.15.96:0/2815593405 conn(0x7fbeb40fd390 msgr2=0x7fbeb4105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 shutdown_connections 2024-04-25T17:10:12.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.717+0000 7fbeb8e13640 1 -- 172.21.15.96:0/2815593405 wait complete. 2024-04-25T17:10:12.728 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T17:10:12.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 -- 172.21.15.96:0/477957777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c1044c0 msgr2=0x7f749c10ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:12.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 --2- 172.21.15.96:0/477957777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c1044c0 0x7f749c10ac00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7494009ca0 tx=0x7f749402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 -- 172.21.15.96:0/477957777 shutdown_connections 2024-04-25T17:10:12.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 --2- 172.21.15.96:0/477957777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c1044c0 0x7f749c10ac00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 --2- 172.21.15.96:0/477957777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f749c103980 0x7f749c103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 --2- 172.21.15.96:0/477957777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c102fa0 0x7f749c1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 -- 172.21.15.96:0/477957777 >> 172.21.15.96:0/477957777 conn(0x7f749c0fd3c0 msgr2=0x7f749c0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:12.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 -- 172.21.15.96:0/477957777 shutdown_connections 2024-04-25T17:10:12.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.878+0000 7f74a196c640 1 -- 172.21.15.96:0/477957777 wait complete. 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 Processor -- start 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 -- start start 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c102fa0 0x7f749c1022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f749c103980 0x7f749c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c1044c0 0x7f749c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f749c10e570 con 0x7f749c102fa0 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f749c10e3f0 con 0x7f749c1044c0 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f74a196c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f749c10e6f0 con 0x7f749c103980 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c102fa0 0x7f749c1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:12.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c102fa0 0x7f749c1022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55588/0 (socket says 172.21.15.96:55588) 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 -- 172.21.15.96:0/1551730835 learned_addr learned my addr 172.21.15.96:0/1551730835 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749a7fc640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f749c103980 0x7f749c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749b7fe640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c1044c0 0x7f749c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 -- 172.21.15.96:0/1551730835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f749c103980 msgr2=0x7f749c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f749c103980 0x7f749c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 -- 172.21.15.96:0/1551730835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c1044c0 msgr2=0x7f749c1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c1044c0 0x7f749c1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.879+0000 7f749affd640 1 -- 172.21.15.96:0/1551730835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c1a1ca0 con 0x7f749c102fa0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.880+0000 7f749affd640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c102fa0 0x7f749c1022a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f748c00d950 tx=0x7f748c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.880+0000 7f74a096a640 1 -- 172.21.15.96:0/1551730835 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c017070 con 0x7f749c102fa0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.880+0000 7f74a096a640 1 -- 172.21.15.96:0/1551730835 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c009c30 con 0x7f749c102fa0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.880+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c1a1f90 con 0x7f749c102fa0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.880+0000 7f74a096a640 1 -- 172.21.15.96:0/1551730835 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0047a0 con 0x7f749c102fa0 2024-04-25T17:10:12.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.880+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c1a23f0 con 0x7f749c102fa0 2024-04-25T17:10:12.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.881+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c050770 con 0x7f749c102fa0 2024-04-25T17:10:12.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.883+0000 7f74a096a640 1 -- 172.21.15.96:0/1551730835 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f748c004020 con 0x7f749c102fa0 2024-04-25T17:10:12.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.884+0000 7f74a096a640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f74840afb30 0x7f74840b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:12.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.884+0000 7f749a7fc640 1 -- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f74840afb30 msgr2=0x7f74840b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:12.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.884+0000 7f74a096a640 1 -- 172.21.15.96:0/1551730835 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f748c0ded00 con 0x7f749c102fa0 2024-04-25T17:10:12.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.884+0000 7f749a7fc640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f74840afb30 0x7f74840b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:10:12.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:12.887+0000 7f74a096a640 1 -- 172.21.15.96:0/1551730835 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f749c050770 con 0x7f749c102fa0 2024-04-25T17:10:12.975 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:12.636+0000 7fcda5670200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:10:12.975 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:12.730+0000 7fcda5670200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:10:13.008 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:12.909+0000 7f475c85a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:10:13.019 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:12.673+0000 7fae9ecdd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T17:10:13.019 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:12.770+0000 7fae9ecdd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:10:13.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.071+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f749c1099a0 con 0x7f749c102fa0 2024-04-25T17:10:13.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.075+0000 7f74a096a640 1 -- 172.21.15.96:0/1551730835 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f749c1099a0 con 0x7f749c102fa0 2024-04-25T17:10:13.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f74840afb30 msgr2=0x7f74840b2040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:10:13.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f74840afb30 0x7f74840b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c102fa0 msgr2=0x7f749c1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:13.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c102fa0 0x7f749c1022a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f748c00d950 tx=0x7f748c00de10 comp rx=0 tx=0).stop 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 shutdown_connections 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749c1044c0 0x7f749c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f74840afb30 0x7f74840b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f749c103980 0x7f749c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 --2- 172.21.15.96:0/1551730835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f749c102fa0 0x7f749c1022a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 >> 172.21.15.96:0/1551730835 conn(0x7f749c0fd3c0 msgr2=0x7f749c106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 shutdown_connections 2024-04-25T17:10:13.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.078+0000 7f74a196c640 1 -- 172.21.15.96:0/1551730835 wait complete. 2024-04-25T17:10:13.089 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T17:10:13.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 -- 172.21.15.96:0/3245299839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 msgr2=0x7f8298103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 --2- 172.21.15.96:0/3245299839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 0x7f8298103e20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8288009900 tx=0x7f828802a3f0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 -- 172.21.15.96:0/3245299839 shutdown_connections 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 --2- 172.21.15.96:0/3245299839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82981044b0 0x7f829810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 --2- 172.21.15.96:0/3245299839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 0x7f8298103e20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 --2- 172.21.15.96:0/3245299839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8298102f90 0x7f82981033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 -- 172.21.15.96:0/3245299839 >> 172.21.15.96:0/3245299839 conn(0x7f82980fd390 msgr2=0x7f82980ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 -- 172.21.15.96:0/3245299839 shutdown_connections 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.201+0000 7f829f80d640 1 -- 172.21.15.96:0/3245299839 wait complete. 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 Processor -- start 2024-04-25T17:10:13.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 -- start start 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298102f90 0x7f8298102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 0x7f829819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82981044b0 0x7f82981a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f829810e580 con 0x7f8298103970 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f829810e400 con 0x7f8298102f90 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829f80d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f829810e700 con 0x7f82981044b0 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829cda8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 0x7f829819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829cda8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 0x7f829819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55606/0 (socket says 172.21.15.96:55606) 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829cda8640 1 -- 172.21.15.96:0/430073318 learned_addr learned my addr 172.21.15.96:0/430073318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829d5a9640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298102f90 0x7f8298102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.202+0000 7f829ddaa640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82981044b0 0x7f82981a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829cda8640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82981044b0 msgr2=0x7f82981a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:13.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829cda8640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82981044b0 0x7f82981a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829cda8640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298102f90 msgr2=0x7f8298102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829cda8640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8298102f90 0x7f8298102250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829cda8640 1 -- 172.21.15.96:0/430073318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82981a1d80 con 0x7f8298103970 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829cda8640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 0x7f829819eff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8288006010 tx=0x7f8288031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828803f030 con 0x7f8298103970 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8288003aa0 con 0x7f8298103970 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82981a2010 con 0x7f8298103970 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288035070 con 0x7f8298103970 2024-04-25T17:10:13.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.203+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82981a2500 con 0x7f8298103970 2024-04-25T17:10:13.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.204+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8298107930 con 0x7f8298103970 2024-04-25T17:10:13.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.206+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 147537+0+0 (secure 0 0 0) 0x7f8288042030 con 0x7f8298103970 2024-04-25T17:10:13.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.206+0000 7f828e7fc640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 0x7f82840b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:13.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.206+0000 7f829d5a9640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 msgr2=0x7f82840b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:13.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.206+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8240+0+0 (secure 0 0 0) 0x7f82880ff850 con 0x7f8298103970 2024-04-25T17:10:13.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.207+0000 7f829d5a9640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 0x7f82840b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T17:10:13.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.215+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82880c7de0 con 0x7f8298103970 2024-04-25T17:10:13.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:12.975+0000 7fcda5670200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:10:13.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:13.110+0000 7fcda5670200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:10:13.233 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:13.207+0000 7fcda5670200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:10:13.280 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:13.018+0000 7fae9ecdd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:10:13.280 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:13.155+0000 7fae9ecdd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:10:13.280 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:13.253+0000 7fae9ecdd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:10:13.281 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:13.009+0000 7f475c85a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T17:10:13.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.408+0000 7f829d5a9640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 msgr2=0x7f82840b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:13.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.408+0000 7f829d5a9640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 0x7f82840b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T17:10:13.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.422+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 --> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8298062fe0 con 0x7f82840afb30 2024-04-25T17:10:13.627 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:13.324+0000 7fcda5670200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:10:13.643 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:13.282+0000 7f475c85a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T17:10:13.643 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:13.425+0000 7f475c85a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T17:10:13.643 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:13.523+0000 7f475c85a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T17:10:13.679 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:13.371+0000 7fae9ecdd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:10:13.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.808+0000 7f829d5a9640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 msgr2=0x7f82840b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:13.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:13.808+0000 7f829d5a9640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 0x7f82840b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T17:10:13.938 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:13.643+0000 7f475c85a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T17:10:13.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2815593405' entity='client.admin' 2024-04-25T17:10:13.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1551730835' entity='client.admin' 2024-04-25T17:10:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:13 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2815593405' entity='client.admin' 2024-04-25T17:10:13.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:13 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1551730835' entity='client.admin' 2024-04-25T17:10:13.983 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:13.627+0000 7fcda5670200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:10:14.029 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:13.678+0000 7fae9ecdd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:10:14.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:13 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2815593405' entity='client.admin' 2024-04-25T17:10:14.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:13 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1551730835' entity='client.admin' 2024-04-25T17:10:14.438 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:13.947+0000 7f475c85a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T17:10:14.483 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:14.232+0000 7fcda5670200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:10:14.483 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y[33989]: 2024-04-25T17:10:14.454+0000 7fcda5670200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:10:14.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:14.609+0000 7f829d5a9640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 msgr2=0x7f82840b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3530143863 2024-04-25T17:10:14.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:14.609+0000 7f829d5a9640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 0x7f82840b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T17:10:14.714 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:14.289+0000 7fae9ecdd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:10:14.714 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z[35069]: 2024-04-25T17:10:14.510+0000 7fae9ecdd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:10:14.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:14.729+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 20) ==== 147537+0+0 (secure 0 0 0) 0x7f82880c4070 con 0x7f8298103970 2024-04-25T17:10:14.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:14.798+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 21) ==== 147205+0+0 (secure 0 0 0) 0x7f82880fd3d0 con 0x7f8298103970 2024-04-25T17:10:14.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:14.799+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 msgr2=0x7f82840b2040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T17:10:14.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:14.799+0000 7f828e7fc640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/3530143863,v1:172.21.15.96:6801/3530143863] conn(0x7f82840afb30 0x7f82840b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:14.931 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:14.559+0000 7f475c85a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T17:10:14.931 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:14.786+0000 7f475c85a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: Standby manager daemon y restarted 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: Standby manager daemon y started 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: Standby manager daemon z restarted 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: Standby manager daemon z started 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T17:10:14.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:14 smithi096 ceph-mon[29131]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: Standby manager daemon y restarted 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: Standby manager daemon y started 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: Standby manager daemon z restarted 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: Standby manager daemon z started 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T17:10:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:14 smithi097 ceph-mon[32602]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: Standby manager daemon y restarted 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: Standby manager daemon y started 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.97:0/545970505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: Standby manager daemon z restarted 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: Standby manager daemon z started 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T17:10:15.018 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:14 smithi171 ceph-mon[33189]: from='mgr.? 172.21.15.171:0/3863144812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.046+0000 7f46bb761700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.046+0000 7f46bb761700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.046+0000 7f46bb761700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.046+0000 7f46bb761700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.046+0000 7f46bb761700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.047+0000 7f46be767700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.047+0000 7f46be767700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.047+0000 7f46be767700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.047+0000 7f46be767700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.194 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:10:15.047+0000 7f46be767700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:10:15.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.807+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mon.0 v2:172.21.15.96:3300/0 9 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f82880883b0 con 0x7f8298103970 2024-04-25T17:10:15.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.808+0000 7f828e7fc640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8284039580 0x7f8284039990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:15.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.808+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 --> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8298062fe0 con 0x7f8284039580 2024-04-25T17:10:15.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.811+0000 7f829d5a9640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8284039580 0x7f8284039990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:15.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.814+0000 7f829d5a9640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8284039580 0x7f8284039990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f829806a330 tx=0x7f8294005f90 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:15.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.843+0000 7f828e7fc640 1 -- 172.21.15.96:0/430073318 <== mgr.15261 v2:172.21.15.96:6800/1594342167 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8298062fe0 con 0x7f8284039580 2024-04-25T17:10:15.843 INFO:teuthology.orchestra.run.smithi096.stdout:Schedule set for path / 2024-04-25T17:10:15.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8284039580 msgr2=0x7f8284039990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:15.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8284039580 0x7f8284039990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f829806a330 tx=0x7f8294005f90 comp rx=0 tx=0).stop 2024-04-25T17:10:15.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 msgr2=0x7f829819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:15.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 --2- 172.21.15.96:0/430073318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8298103970 0x7f829819eff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8288006010 tx=0x7f8288031a50 comp rx=0 tx=0).stop 2024-04-25T17:10:15.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829ddaa640 1 -- 172.21.15.96:0/430073318 reap_dead start 2024-04-25T17:10:15.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 shutdown_connections 2024-04-25T17:10:15.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 >> 172.21.15.96:0/430073318 conn(0x7f82980fd390 msgr2=0x7f8298104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:15.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 shutdown_connections 2024-04-25T17:10:15.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.845+0000 7f829f80d640 1 -- 172.21.15.96:0/430073318 wait complete. 2024-04-25T17:10:15.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T17:10:15.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.968+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/2400484688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 msgr2=0x7fd4cc105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.968+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/2400484688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc105190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009900 tx=0x7fd4b802a410 comp rx=0 tx=0).stop 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.968+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/2400484688 shutdown_connections 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.968+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/2400484688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4cc109a80 0x7fd4cc10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/2400484688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc105190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/2400484688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4cc103b40 0x7fd4cc104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/2400484688 >> 172.21.15.96:0/2400484688 conn(0x7fd4cc0fed10 msgr2=0x7fd4cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/2400484688 shutdown_connections 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/2400484688 wait complete. 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 Processor -- start 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 -- start start 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4cc103b40 0x7fd4cc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:15.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4cc109a80 0x7fd4cc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc10f660 con 0x7fd4cc104d80 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc10f4e0 con 0x7fd4cc103b40 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.969+0000 7fd4d2fe3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc10f7e0 con 0x7fd4cc109a80 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55722/0 (socket says 172.21.15.96:55722) 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 -- 172.21.15.96:0/3316639445 learned_addr learned my addr 172.21.15.96:0/3316639445 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4d0d7f640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4cc103b40 0x7fd4cc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4d1580640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4cc109a80 0x7fd4cc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 -- 172.21.15.96:0/3316639445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4cc109a80 msgr2=0x7fd4cc1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4cc109a80 0x7fd4cc1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 -- 172.21.15.96:0/3316639445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4cc103b40 msgr2=0x7fd4cc19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:15.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4cc103b40 0x7fd4cc19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:15.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 -- 172.21.15.96:0/3316639445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc1a1d80 con 0x7fd4cc104d80 2024-04-25T17:10:15.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4cbfff640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc19a780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8006010 tx=0x7fd4b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:15.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4c9ffb640 1 -- 172.21.15.96:0/3316639445 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b803f030 con 0x7fd4cc104d80 2024-04-25T17:10:15.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4c9ffb640 1 -- 172.21.15.96:0/3316639445 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b8031910 con 0x7fd4cc104d80 2024-04-25T17:10:15.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc1a2010 con 0x7fd4cc104d80 2024-04-25T17:10:15.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4c9ffb640 1 -- 172.21.15.96:0/3316639445 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8035070 con 0x7fd4cc104d80 2024-04-25T17:10:15.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.970+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc1a2500 con 0x7fd4cc104d80 2024-04-25T17:10:15.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.971+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc108890 con 0x7fd4cc104d80 2024-04-25T17:10:15.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.973+0000 7fd4c9ffb640 1 -- 172.21.15.96:0/3316639445 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7fd4b8042030 con 0x7fd4cc104d80 2024-04-25T17:10:15.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.973+0000 7fd4c9ffb640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd4bc0af8e0 0x7fd4bc0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:15.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.973+0000 7fd4c9ffb640 1 -- 172.21.15.96:0/3316639445 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7fd4b80ff540 con 0x7fd4cc104d80 2024-04-25T17:10:15.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.973+0000 7fd4d0d7f640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd4bc0af8e0 0x7fd4bc0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:15.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.974+0000 7fd4d0d7f640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd4bc0af8e0 0x7fd4bc0b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc104740 tx=0x7fd4c0005f90 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:15.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:15.977+0000 7fd4c9ffb640 1 -- 172.21.15.96:0/3316639445 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4b80c8300 con 0x7fd4cc104d80 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: Active manager daemon x restarted 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: Activating manager daemon x 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: mgrmap e21: x(active, starting, since 0.0133227s), standbys: z, y 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T17:10:15.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:10:15.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: Manager daemon x is now available 2024-04-25T17:10:15.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:10:15.987 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T17:10:15.987 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T17:10:15.987 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: [25/Apr/2024:17:10:15] ENGINE Bus STARTING 2024-04-25T17:10:15.987 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: [25/Apr/2024:17:10:15] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T17:10:15.987 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: [25/Apr/2024:17:10:15] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T17:10:15.987 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:15 smithi097 ceph-mon[32602]: [25/Apr/2024:17:10:15] ENGINE Bus STARTED 2024-04-25T17:10:16.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: Active manager daemon x restarted 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: Activating manager daemon x 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: mgrmap e21: x(active, starting, since 0.0133227s), standbys: z, y 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T17:10:16.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:10:16.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: Manager daemon x is now available 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: [25/Apr/2024:17:10:15] ENGINE Bus STARTING 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: [25/Apr/2024:17:10:15] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: [25/Apr/2024:17:10:15] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T17:10:16.033 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:15 smithi171 ceph-mon[33189]: [25/Apr/2024:17:10:15] ENGINE Bus STARTED 2024-04-25T17:10:16.096 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-25T17:10:16.096 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: Active manager daemon x restarted 2024-04-25T17:10:16.096 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: Activating manager daemon x 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: mgrmap e21: x(active, starting, since 0.0133227s), standbys: z, y 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T17:10:16.097 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T17:10:16.098 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: Manager daemon x is now available 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: [25/Apr/2024:17:10:15] ENGINE Bus STARTING 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: [25/Apr/2024:17:10:15] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T17:10:16.099 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: [25/Apr/2024:17:10:15] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T17:10:16.100 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:15 smithi096 ceph-mon[29131]: [25/Apr/2024:17:10:15] ENGINE Bus STARTED 2024-04-25T17:10:16.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.185+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 --> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fd4cc063140 con 0x7fd4bc0af8e0 2024-04-25T17:10:16.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.211+0000 7fd4c9ffb640 1 -- 172.21.15.96:0/3316639445 <== mgr.15261 v2:172.21.15.96:6800/1594342167 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fd4cc063140 con 0x7fd4bc0af8e0 2024-04-25T17:10:16.212 INFO:teuthology.orchestra.run.smithi096.stdout:Retention added to path / 2024-04-25T17:10:16.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd4bc0af8e0 msgr2=0x7fd4bc0b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd4bc0af8e0 0x7fd4bc0b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc104740 tx=0x7fd4c0005f90 comp rx=0 tx=0).stop 2024-04-25T17:10:16.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 msgr2=0x7fd4cc19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc19a780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8006010 tx=0x7fd4b80042f0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 shutdown_connections 2024-04-25T17:10:16.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4cc109a80 0x7fd4cc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4cc104d80 0x7fd4cc19a780 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd4bc0af8e0 0x7fd4bc0b1df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 --2- 172.21.15.96:0/3316639445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4cc103b40 0x7fd4cc19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 >> 172.21.15.96:0/3316639445 conn(0x7fd4cc0fed10 msgr2=0x7fd4cc105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:16.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 shutdown_connections 2024-04-25T17:10:16.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.214+0000 7fd4d2fe3640 1 -- 172.21.15.96:0/3316639445 wait complete. 2024-04-25T17:10:16.226 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T17:10:16.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 -- 172.21.15.96:0/2993316540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec104240 msgr2=0x7f2eec106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 --2- 172.21.15.96:0/2993316540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec104240 0x7f2eec106660 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009900 tx=0x7f2edc02a410 comp rx=0 tx=0).stop 2024-04-25T17:10:16.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 -- 172.21.15.96:0/2993316540 shutdown_connections 2024-04-25T17:10:16.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 --2- 172.21.15.96:0/2993316540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec1097a0 0x7f2eec10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 --2- 172.21.15.96:0/2993316540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eec106c30 0x7f2eec109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 --2- 172.21.15.96:0/2993316540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec104240 0x7f2eec106660 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 -- 172.21.15.96:0/2993316540 >> 172.21.15.96:0/2993316540 conn(0x7f2eec0fdc00 msgr2=0x7f2eec1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 -- 172.21.15.96:0/2993316540 shutdown_connections 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.336+0000 7f2ef1805640 1 -- 172.21.15.96:0/2993316540 wait complete. 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 Processor -- start 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 -- start start 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec104240 0x7f2eec19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec106c30 0x7f2eec19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eec1097a0 0x7f2eec1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f600 con 0x7f2eec106c30 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f480 con 0x7f2eec1097a0 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef1805640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f780 con 0x7f2eec104240 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec106c30 0x7f2eec19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec106c30 0x7f2eec19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55744/0 (socket says 172.21.15.96:55744) 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2eeaffd640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec104240 0x7f2eec19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 -- 172.21.15.96:0/1555732435 learned_addr learned my addr 172.21.15.96:0/1555732435 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:16.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2eeb7fe640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eec1097a0 0x7f2eec1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 -- 172.21.15.96:0/1555732435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec104240 msgr2=0x7f2eec19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec104240 0x7f2eec19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 -- 172.21.15.96:0/1555732435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eec1097a0 msgr2=0x7f2eec1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eec1097a0 0x7f2eec1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 -- 172.21.15.96:0/1555732435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec1a1da0 con 0x7f2eec106c30 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2eeb7fe640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eec1097a0 0x7f2eec1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ee3fff640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec106c30 0x7f2eec19a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400d950 tx=0x7f2ed400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.337+0000 7f2ef0803640 1 -- 172.21.15.96:0/1555732435 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4017070 con 0x7f2eec106c30 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.338+0000 7f2ef0803640 1 -- 172.21.15.96:0/1555732435 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed4009c30 con 0x7f2eec106c30 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.338+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec1a2090 con 0x7f2eec106c30 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.338+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec1a25e0 con 0x7f2eec106c30 2024-04-25T17:10:16.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.338+0000 7f2ef0803640 1 -- 172.21.15.96:0/1555732435 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400f620 con 0x7f2eec106c30 2024-04-25T17:10:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.338+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec0ff720 con 0x7f2eec106c30 2024-04-25T17:10:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.340+0000 7f2ef0803640 1 -- 172.21.15.96:0/1555732435 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f2ed4010460 con 0x7f2eec106c30 2024-04-25T17:10:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.340+0000 7f2ef0803640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f2ed80af8e0 0x7f2ed80b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.341+0000 7f2eeaffd640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f2ed80af8e0 0x7f2ed80b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.341+0000 7f2ef0803640 1 -- 172.21.15.96:0/1555732435 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f2ed40de590 con 0x7f2eec106c30 2024-04-25T17:10:16.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.341+0000 7f2eeaffd640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f2ed80af8e0 0x7f2ed80b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009230 tx=0x7f2edc031400 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:16.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.344+0000 7f2ef0803640 1 -- 172.21.15.96:0/1555732435 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ed40a7350 con 0x7f2eec106c30 2024-04-25T17:10:16.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.550+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 --> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2eec062fe0 con 0x7f2ed80af8e0 2024-04-25T17:10:16.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.559+0000 7f2ef0803640 1 -- 172.21.15.96:0/1555732435 <== mgr.15261 v2:172.21.15.96:6800/1594342167 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2eec062fe0 con 0x7f2ed80af8e0 2024-04-25T17:10:16.559 INFO:teuthology.orchestra.run.smithi096.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T17:10:15", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T17:10:16.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f2ed80af8e0 msgr2=0x7f2ed80b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f2ed80af8e0 0x7f2ed80b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009230 tx=0x7f2edc031400 comp rx=0 tx=0).stop 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec106c30 msgr2=0x7f2eec19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec106c30 0x7f2eec19a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400d950 tx=0x7f2ed400de10 comp rx=0 tx=0).stop 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 shutdown_connections 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f2ed80af8e0 0x7f2ed80b1df0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eec1097a0 0x7f2eec1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eec106c30 0x7f2eec19a7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 --2- 172.21.15.96:0/1555732435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2eec104240 0x7f2eec19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 >> 172.21.15.96:0/1555732435 conn(0x7f2eec0fdc00 msgr2=0x7f2eec106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 shutdown_connections 2024-04-25T17:10:16.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.561+0000 7f2ef1805640 1 -- 172.21.15.96:0/1555732435 wait complete. 2024-04-25T17:10:16.573 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T17:10:16.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 -- 172.21.15.96:0/4170010300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102a40 msgr2=0x7f90ac103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 --2- 172.21.15.96:0/4170010300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102a40 0x7f90ac103740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f909c009900 tx=0x7f909c02a410 comp rx=0 tx=0).stop 2024-04-25T17:10:16.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 -- 172.21.15.96:0/4170010300 shutdown_connections 2024-04-25T17:10:16.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 --2- 172.21.15.96:0/4170010300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac108980 0x7f90ac10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 --2- 172.21.15.96:0/4170010300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90ac103c80 0x7f90ac104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 --2- 172.21.15.96:0/4170010300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102a40 0x7f90ac103740 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 -- 172.21.15.96:0/4170010300 >> 172.21.15.96:0/4170010300 conn(0x7f90ac0fdc30 msgr2=0x7f90ac1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 -- 172.21.15.96:0/4170010300 shutdown_connections 2024-04-25T17:10:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.702+0000 7f90b1e34640 1 -- 172.21.15.96:0/4170010300 wait complete. 2024-04-25T17:10:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 Processor -- start 2024-04-25T17:10:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 -- start start 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac102a40 0x7f90ac10d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90ac103c80 0x7f90ac10b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac108980 0x7f90ac10bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac10f670 con 0x7f90ac108980 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac10f4f0 con 0x7f90ac102a40 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.703+0000 7f90b1e34640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac10f7f0 con 0x7f90ac103c80 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac108980 0x7f90ac10bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac102a40 0x7f90ac10d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac108980 0x7f90ac10bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55754/0 (socket says 172.21.15.96:55754) 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac102a40 0x7f90ac10d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:39924/0 (socket says 172.21.15.96:39924) 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 -- 172.21.15.96:0/2145056802 learned_addr learned my addr 172.21.15.96:0/2145056802 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90aaffd640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90ac103c80 0x7f90ac10b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 -- 172.21.15.96:0/2145056802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90ac103c80 msgr2=0x7f90ac10b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90ac103c80 0x7f90ac10b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 -- 172.21.15.96:0/2145056802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac102a40 msgr2=0x7f90ac10d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac102a40 0x7f90ac10d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 -- 172.21.15.96:0/2145056802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac10c620 con 0x7f90ac108980 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90ab7fe640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac102a40 0x7f90ac10d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90abfff640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac108980 0x7f90ac10bef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90a400eb10 tx=0x7f90a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4004b20 con 0x7f90ac108980 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac1a4320 con 0x7f90ac108980 2024-04-25T17:10:16.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a4004cc0 con 0x7f90ac108980 2024-04-25T17:10:16.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a47e0 con 0x7f90ac108980 2024-04-25T17:10:16.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.704+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4010670 con 0x7f90ac108980 2024-04-25T17:10:16.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.705+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac107790 con 0x7f90ac108980 2024-04-25T17:10:16.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.707+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 147332+0+0 (secure 0 0 0) 0x7f90a4005000 con 0x7f90ac108980 2024-04-25T17:10:16.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.708+0000 7f90a8ff9640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f90940af8e0 0x7f90940b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:16.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.708+0000 7f90ab7fe640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f90940af8e0 0x7f90940b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:16.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.708+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f90a40df0a0 con 0x7f90ac108980 2024-04-25T17:10:16.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.709+0000 7f90ab7fe640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f90940af8e0 0x7f90940b1df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f909c005ab0 tx=0x7f909c031400 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:16.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.712+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90a4053ff0 con 0x7f90ac108980 2024-04-25T17:10:16.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.817+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 23) ==== 147640+0+0 (secure 0 0 0) 0x7f90a40a39f0 con 0x7f90ac108980 2024-04-25T17:10:16.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:16 smithi096 ceph-mon[29131]: mgrmap e22: x(active, since 1.02237s), standbys: z, y 2024-04-25T17:10:16.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:16 smithi096 ceph-mon[29131]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:16.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:16 smithi096 ceph-mon[29131]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:16.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:16 smithi096 ceph-mon[29131]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:16.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:16 smithi096 ceph-mon[29131]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:16.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.965+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 --> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f90ac062fe0 con 0x7f90940af8e0 2024-04-25T17:10:16.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.973+0000 7f90a8ff9640 1 -- 172.21.15.96:0/2145056802 <== mgr.15261 v2:172.21.15.96:6800/1594342167 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f90ac062fe0 con 0x7f90940af8e0 2024-04-25T17:10:16.973 INFO:teuthology.orchestra.run.smithi096.stdout:/ 1m 6m3h 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f90940af8e0 msgr2=0x7f90940b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f90940af8e0 0x7f90940b1df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f909c005ab0 tx=0x7f909c031400 comp rx=0 tx=0).stop 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac108980 msgr2=0x7f90ac10bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac108980 0x7f90ac10bef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90a400eb10 tx=0x7f90a400c2d0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 shutdown_connections 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac108980 0x7f90ac10bef0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90ac103c80 0x7f90ac10b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f90940af8e0 0x7f90940b1df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.975+0000 7f90b1e34640 1 --2- 172.21.15.96:0/2145056802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac102a40 0x7f90ac10d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.976+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 >> 172.21.15.96:0/2145056802 conn(0x7f90ac0fdc30 msgr2=0x7f90ac104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.976+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 shutdown_connections 2024-04-25T17:10:16.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:10:16.976+0000 7f90b1e34640 1 -- 172.21.15.96:0/2145056802 wait complete. 2024-04-25T17:10:16.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T17:10:17.011 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T17:10:17.021 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T17:10:17.032 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:10:17.032 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T17:10:17.055 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T17:10:17.066 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T17:10:17.066 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T17:10:17.066 INFO:tasks.workunit:timeout=3h 2024-04-25T17:10:17.067 INFO:tasks.workunit:cleanup=True 2024-04-25T17:10:17.067 DEBUG:teuthology.orchestra.run.smithi096:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:17.082 INFO:teuthology.orchestra.run.smithi096.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:17.082 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T17:10:17.082 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T17:10:17.083 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T17:10:17.083 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T17:10:17.083 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 17:08:16.853268116 +0000 2024-04-25T17:10:17.083 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 17:08:16.853268116 +0000 2024-04-25T17:10:17.083 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 17:10:12.031032121 +0000 2024-04-25T17:10:17.083 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 17:08:16.853268116 +0000 2024-04-25T17:10:17.083 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T17:10:17.083 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T17:10:17.152 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T17:10:17.207 INFO:tasks.workunit.client.0.smithi096.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T17:10:17.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:16 smithi097 ceph-mon[32602]: mgrmap e22: x(active, since 1.02237s), standbys: z, y 2024-04-25T17:10:17.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:16 smithi097 ceph-mon[32602]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:17.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:16 smithi097 ceph-mon[32602]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:17.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:16 smithi097 ceph-mon[32602]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:17.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:16 smithi097 ceph-mon[32602]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:17.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:16 smithi171 ceph-mon[33189]: mgrmap e22: x(active, since 1.02237s), standbys: z, y 2024-04-25T17:10:17.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:16 smithi171 ceph-mon[33189]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:17.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:16 smithi171 ceph-mon[33189]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:17.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:16 smithi171 ceph-mon[33189]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:17.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:16 smithi171 ceph-mon[33189]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:18.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:17 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.233 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:17 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:17 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:10:19.282 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:10:19.283 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:10:19.283 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T17:10:19.283 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:10:19.283 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:10:19.283 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.283 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.283 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:19 smithi171 ceph-mon[33189]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T17:10:19.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T17:10:19.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T17:10:19.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:10:19.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:10:19.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:10:19.442 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T17:10:19.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:10:19.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:10:19.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.443 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:19 smithi096 ceph-mon[29131]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T17:10:19.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:10:19.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:10:19.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:10:19.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T17:10:19.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T17:10:19.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-25T17:10:19.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:19.487 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:19 smithi097 ceph-mon[32602]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T17:10:20.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:10:20.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:10:20.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:10:20.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:20.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T17:10:20.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.533 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.533 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:10:20.533 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:10:20.533 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:10:20.533 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: Updating smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: Updating smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: Updating smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:10:20.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:10:22.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:22 smithi171 ceph-mon[33189]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 90 KiB/s wr, 21 op/s 2024-04-25T17:10:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:22 smithi096 ceph-mon[29131]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 90 KiB/s wr, 21 op/s 2024-04-25T17:10:22.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:22 smithi097 ceph-mon[32602]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 90 KiB/s wr, 21 op/s 2024-04-25T17:10:24.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:24 smithi171 ceph-mon[33189]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 132 KiB/s wr, 32 op/s 2024-04-25T17:10:24.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:24 smithi096 ceph-mon[29131]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 132 KiB/s wr, 32 op/s 2024-04-25T17:10:24.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:24 smithi097 ceph-mon[32602]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 132 KiB/s wr, 32 op/s 2024-04-25T17:10:26.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:26 smithi171 ceph-mon[33189]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 KiB/s wr, 25 op/s 2024-04-25T17:10:26.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:26 smithi096 ceph-mon[29131]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 KiB/s wr, 25 op/s 2024-04-25T17:10:26.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:26 smithi097 ceph-mon[32602]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 102 KiB/s wr, 25 op/s 2024-04-25T17:10:28.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:28 smithi171 ceph-mon[33189]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 21 op/s 2024-04-25T17:10:28.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:28 smithi096 ceph-mon[29131]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 21 op/s 2024-04-25T17:10:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:28 smithi097 ceph-mon[32602]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 84 KiB/s wr, 21 op/s 2024-04-25T17:10:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.192+0000 7f3cdada8640 1 -- 172.21.15.96:0/3495935810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd410e580 msgr2=0x7f3cd41109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 --2- 172.21.15.96:0/3495935810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd410e580 0x7f3cd41109a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009ca0 tx=0x7f3ccc02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 -- 172.21.15.96:0/3495935810 shutdown_connections 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 --2- 172.21.15.96:0/3495935810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd410e580 0x7f3cd41109a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 --2- 172.21.15.96:0/3495935810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cd4102080 0x7f3cd4102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 --2- 172.21.15.96:0/3495935810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cd4109bd0 0x7f3cd4101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 -- 172.21.15.96:0/3495935810 >> 172.21.15.96:0/3495935810 conn(0x7f3cd40fd8f0 msgr2=0x7f3cd40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 -- 172.21.15.96:0/3495935810 shutdown_connections 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 -- 172.21.15.96:0/3495935810 wait complete. 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.193+0000 7f3cdada8640 1 Processor -- start 2024-04-25T17:10:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 -- start start 2024-04-25T17:10:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cd4102080 0x7f3cd419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd4109bd0 0x7f3cd419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cd410e580 0x7f3cd419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4113d40 con 0x7f3cd4109bd0 2024-04-25T17:10:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4113bc0 con 0x7f3cd4102080 2024-04-25T17:10:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4113ec0 con 0x7f3cd410e580 2024-04-25T17:10:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd4109bd0 0x7f3cd419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd4109bd0 0x7f3cd419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49044/0 (socket says 172.21.15.96:49044) 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 -- 172.21.15.96:0/1715769019 learned_addr learned my addr 172.21.15.96:0/1715769019 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd9345640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cd410e580 0x7f3cd419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cd4102080 0x7f3cd419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 -- 172.21.15.96:0/1715769019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cd410e580 msgr2=0x7f3cd419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cd410e580 0x7f3cd419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 -- 172.21.15.96:0/1715769019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cd4102080 msgr2=0x7f3cd419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cd4102080 0x7f3cd419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 -- 172.21.15.96:0/1715769019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd419f540 con 0x7f3cd4109bd0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd4109bd0 0x7f3cd419ac40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005fd0 tx=0x7f3cc000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd1ffb640 1 -- 172.21.15.96:0/1715769019 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0017070 con 0x7f3cd4109bd0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cd1ffb640 1 -- 172.21.15.96:0/1715769019 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc0009c30 con 0x7f3cd4109bd0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.194+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd419f830 con 0x7f3cd4109bd0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.195+0000 7f3cd1ffb640 1 -- 172.21.15.96:0/1715769019 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0003d30 con 0x7f3cd4109bd0 2024-04-25T17:10:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.195+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd41a71b0 con 0x7f3cd4109bd0 2024-04-25T17:10:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.197+0000 7f3cd1ffb640 1 -- 172.21.15.96:0/1715769019 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7f3cc0003ed0 con 0x7f3cd4109bd0 2024-04-25T17:10:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.197+0000 7f3ce1c5b740 1 Processor -- start 2024-04-25T17:10:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.197+0000 7f3ce1c5b740 1 -- start start 2024-04-25T17:10:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.197+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b8d1b0 0x55ff93f2caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b642f0 0x55ff93f2d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f2d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff93e93460 con 0x55ff93b642f0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ff93e4d590 con 0x55ff93b8d1b0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ff93e8f420 con 0x55ff93b6c4d0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd1ffb640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f3cc40afce0 0x7f3cc40b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b642f0 0x55ff93f2d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b642f0 0x55ff93f2d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49056/0 (socket says 172.21.15.96:49056) 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 -- 172.21.15.96:0/200093750 learned_addr learned my addr 172.21.15.96:0/200093750 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd9345640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f3cc40afce0 0x7f3cc40b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd9345640 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b8d1b0 0x55ff93f2caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd1ffb640 1 -- 172.21.15.96:0/1715769019 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f3cc00def60 con 0x7f3cd4109bd0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f2d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 -- 172.21.15.96:0/200093750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 msgr2=0x55ff93f2d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f2d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 -- 172.21.15.96:0/200093750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b8d1b0 msgr2=0x55ff93f2caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b8d1b0 0x55ff93f2caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 -- 172.21.15.96:0/200093750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff93e5a0d0 con 0x55ff93b642f0 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f2d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b642f0 0x55ff93f2d030 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009b70 tx=0x7f3cc80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edf91e05447744f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cd9345640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f3cc40afce0 0x7f3cc40b21f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55ff93f29c60 tx=0x7f3ccc0095c0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cbeffd640 1 -- 172.21.15.96:0/200093750 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8009e60 con 0x55ff93b642f0 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.198+0000 7f3cbeffd640 1 -- 172.21.15.96:0/200093750 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc8012900 con 0x55ff93b642f0 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3cbeffd640 1 -- 172.21.15.96:0/200093750 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800fa40 con 0x55ff93b642f0 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/200093750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b642f0 msgr2=0x55ff93f2d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b642f0 0x55ff93f2d030 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009b70 tx=0x7f3cc80117b0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/200093750 shutdown_connections 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f2d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b642f0 0x55ff93f2d030 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/200093750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b8d1b0 0x55ff93f2caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/200093750 >> 172.21.15.96:0/200093750 conn(0x55ff93b81300 msgr2=0x55ff93f292a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/200093750 shutdown_connections 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.199+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/200093750 wait complete. 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 -- start start 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 Processor -- start 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f37190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f3d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ff93f39c20 con 0x55ff93b6c4d0 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ff93e1ab60 con 0x55ff93b642f0 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ff93f39da0 con 0x55ff93b8d1b0 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3ce1c5b740 10 client.0 _ll_register_callbacks cb 0x55ff93e3f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f376d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49066/0 (socket says 172.21.15.96:49066) 2024-04-25T17:10:30.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.200+0000 7f3cd3fff640 1 -- 172.21.15.96:0/4089518607 learned_addr learned my addr 172.21.15.96:0/4089518607 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd9345640 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f3d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f37190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd3fff640 1 -- 172.21.15.96:0/4089518607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 msgr2=0x55ff93f3d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f3d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd3fff640 1 -- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 msgr2=0x55ff93f37190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f37190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd3fff640 1 -- 172.21.15.96:0/4089518607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff93ba5210 con 0x55ff93b6c4d0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f376d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00a4210 tx=0x7f3cc0009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d8cc70ad4b0d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cb3fff640 1 -- 172.21.15.96:0/4089518607 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00ab350 con 0x55ff93b6c4d0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cb3fff640 1 -- 172.21.15.96:0/4089518607 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f3cc00dc8b0 con 0x55ff93b6c4d0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc8012900 con 0x55ff93b6c4d0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cb3fff640 1 -- 172.21.15.96:0/4089518607 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3cc00dcab0 con 0x55ff93b6c4d0 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3ce1c5b740 10 client.15360 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ff93c71620 con 0x55ff93b6c4d0 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.201+0000 7f3cb3fff640 1 -- 172.21.15.96:0/4089518607 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc8012900 con 0x55ff93b6c4d0 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3cb3fff640 1 -- 172.21.15.96:0/4089518607 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f3cc00a2b10 con 0x55ff93b6c4d0 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3ce1c5b740 10 client.15360 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3ce1c5b740 10 client.15360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3ce1c5b740 20 client.15360 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b4bd80 0x55ff93f49620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3ce1c5b740 4 client.15360 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff93ea1070 con 0x55ff93b4bd80 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.202+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b4bd80 0x55ff93f49620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3cb3fff640 10 client.15360 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b4bd80 0x55ff93f49620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9d6815cfc4d18 server_cookie=1259a09aab17d0a2 in_seq=0 out_seq=0 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3cb3fff640 1 -- 172.21.15.96:0/4089518607 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff93ea1070 con 0x55ff93b4bd80 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3cb3fff640 10 client.15360 handle_command_reply: tid=0 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 client.15360 shutdown 2024-04-25T17:10:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 msgr2=0x55ff93f376d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f376d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00a4210 tx=0x7f3cc0009de0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 shutdown_connections 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b4bd80 0x55ff93f49620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f3d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3cb3fff640 0 client.15360 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f376d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/4089518607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f37190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.203+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 >> 172.21.15.96:0/4089518607 conn(0x55ff93b81300 msgr2=0x55ff93f2c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.204+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 shutdown_connections 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.204+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/4089518607 wait complete. 2024-04-25T17:10:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.204+0000 7f3ce1c5b740 20 client.15360 trim_cache size 0 max 16384 2024-04-25T17:10:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.218+0000 7f3ce1c5b740 1 Processor -- start 2024-04-25T17:10:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.218+0000 7f3ce1c5b740 1 -- start start 2024-04-25T17:10:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f30ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f4edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f4f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41ad930 con 0x55ff93b6c4d0 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41b01b0 con 0x55ff93b642f0 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41adfa0 con 0x55ff93b8d1b0 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f4edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f4edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49072/0 (socket says 172.21.15.96:49072) 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 -- 172.21.15.96:0/2954038157 learned_addr learned my addr 172.21.15.96:0/2954038157 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f30ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd9345640 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f4f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 -- 172.21.15.96:0/2954038157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 msgr2=0x55ff93f4f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f4f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 -- 172.21.15.96:0/2954038157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 msgr2=0x55ff93f30ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f30ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 -- 172.21.15.96:0/2954038157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff93f37ea0 con 0x55ff93b6c4d0 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd9345640 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f4f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f30ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f4edd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00a7160 tx=0x7f3cc00a6bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a91da3410d37500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cbe7fc640 1 -- 172.21.15.96:0/2954038157 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00afe00 con 0x55ff93b6c4d0 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cbe7fc640 1 -- 172.21.15.96:0/2954038157 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc00a5e50 con 0x55ff93b6c4d0 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.219+0000 7f3cbe7fc640 1 -- 172.21.15.96:0/2954038157 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00041c0 con 0x55ff93b6c4d0 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/2954038157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 msgr2=0x55ff93f4edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f4edd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00a7160 tx=0x7f3cc00a6bc0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/2954038157 shutdown_connections 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b8d1b0 0x55ff93f4f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b6c4d0 0x55ff93f4edd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/2954038157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f30ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/2954038157 >> 172.21.15.96:0/2954038157 conn(0x55ff93b81300 msgr2=0x55ff93f307b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/2954038157 shutdown_connections 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/2954038157 wait complete. 2024-04-25T17:10:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.220+0000 7f3ce1c5b740 1 -- start start 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 1 Processor -- start 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f43c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f41020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b8d1b0 0x55ff93f41600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41ae670 con 0x55ff93b8d1b0 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41afae0 con 0x55ff93b642f0 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41aed40 con 0x55ff93b6c4d0 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3ce1c5b740 10 client.0 _ll_register_callbacks cb 0x7f3cc00dc490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b8d1b0 0x55ff93f41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b8d1b0 0x55ff93f41600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49078/0 (socket says 172.21.15.96:49078) 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 -- 172.21.15.96:0/278294946 learned_addr learned my addr 172.21.15.96:0/278294946 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f43c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f41020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 -- 172.21.15.96:0/278294946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 msgr2=0x55ff93f41020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f41020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 -- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 msgr2=0x55ff93f43c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f43c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd9345640 1 -- 172.21.15.96:0/278294946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff93f41d60 con 0x55ff93b8d1b0 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd3fff640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.221+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f43c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3cd9345640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b8d1b0 0x55ff93f41600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4102b20 tx=0x7f3ccc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bd54a6f90ca291 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3cb3fff640 1 -- 172.21.15.96:0/278294946 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc05b060 con 0x55ff93b8d1b0 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3cb3fff640 1 -- 172.21.15.96:0/278294946 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f3ccc047070 con 0x55ff93b8d1b0 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc00a5e50 con 0x55ff93b8d1b0 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3cb3fff640 1 -- 172.21.15.96:0/278294946 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3ccc031740 con 0x55ff93b8d1b0 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 10 client.15366 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55ff93fb18c0 con 0x55ff93b8d1b0 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3cb3fff640 1 -- 172.21.15.96:0/278294946 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc00a5e50 con 0x55ff93b8d1b0 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3cb3fff640 1 -- 172.21.15.96:0/278294946 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f3ccc04a040 con 0x55ff93b8d1b0 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 10 client.15366 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 20 client.15366 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b211c0 0x55ff93fb4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 4 client.15366 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.222+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff93e9aee0 con 0x55ff93b211c0 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.223+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b211c0 0x55ff93fb4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.223+0000 7f3cd8b44640 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b211c0 0x55ff93fb4080 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71bdb93f76a4c7c5 server_cookie=28bd4c24a4b0a57 in_seq=0 out_seq=0 2024-04-25T17:10:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.223+0000 7f3cb3fff640 10 client.15366 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3cb3fff640 1 -- 172.21.15.96:0/278294946 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff93e9aee0 con 0x55ff93b211c0 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3cb3fff640 10 client.15366 handle_command_reply: tid=0 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 client.15366 shutdown 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b8d1b0 msgr2=0x55ff93f41600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b8d1b0 0x55ff93f41600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4102b20 tx=0x7f3ccc009360 comp rx=0 tx=0).stop 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 shutdown_connections 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55ff93b211c0 0x55ff93fb4080 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ff93b8d1b0 0x55ff93f41600 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff93b6c4d0 0x55ff93f41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3cb3fff640 0 client.15366 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 --2- 172.21.15.96:0/278294946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ff93b642f0 0x55ff93f43c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 >> 172.21.15.96:0/278294946 conn(0x55ff93b81300 msgr2=0x55ff93f2c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 shutdown_connections 2024-04-25T17:10:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 1 -- 172.21.15.96:0/278294946 wait complete. 2024-04-25T17:10:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.235+0000 7f3ce1c5b740 20 client.15366 trim_cache size 0 max 16384 2024-04-25T17:10:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:10:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:10:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:10:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:10:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f3cc40afce0 msgr2=0x7f3cc40b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f3cc40afce0 0x7f3cc40b21f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x55ff93f29c60 tx=0x7f3ccc0095c0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd4109bd0 msgr2=0x7f3cd419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd4109bd0 0x7f3cd419ac40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005fd0 tx=0x7f3cc000dde0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 shutdown_connections 2024-04-25T17:10:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cd410e580 0x7f3cd419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3cd4109bd0 0x7f3cd419ac40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f3cc40afce0 0x7f3cc40b21f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 --2- 172.21.15.96:0/1715769019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cd4102080 0x7f3cd419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 >> 172.21.15.96:0/1715769019 conn(0x7f3cd40fd8f0 msgr2=0x7f3cd40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 shutdown_connections 2024-04-25T17:10:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:30.236+0000 7f3cdada8640 1 -- 172.21.15.96:0/1715769019 wait complete. 2024-04-25T17:10:30.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:30 smithi171 ceph-mon[33189]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-04-25T17:10:30.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:30 smithi096 ceph-mon[29131]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-04-25T17:10:30.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:30 smithi097 ceph-mon[32602]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-04-25T17:10:32.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:32 smithi171 ceph-mon[33189]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T17:10:32.536 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:32 smithi097 ceph-mon[32602]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T17:10:32.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:32 smithi096 ceph-mon[29131]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 20 op/s 2024-04-25T17:10:34.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:34 smithi171 ceph-mon[33189]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-25T17:10:34.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:34 smithi096 ceph-mon[29131]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-25T17:10:34.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:34 smithi097 ceph-mon[32602]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-25T17:10:36.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:36 smithi171 ceph-mon[33189]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:36.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:36 smithi096 ceph-mon[29131]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:36.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:36 smithi097 ceph-mon[32602]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:38.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:38 smithi171 ceph-mon[33189]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:38.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:38 smithi096 ceph-mon[29131]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:38.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:38 smithi097 ceph-mon[32602]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:40.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:40 smithi096 ceph-mon[29131]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:40.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:40 smithi097 ceph-mon[32602]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:40.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:40 smithi171 ceph-mon[33189]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:42.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:42 smithi096 ceph-mon[29131]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:42.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:42 smithi097 ceph-mon[32602]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:42.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:42 smithi171 ceph-mon[33189]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T17:10:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 -- 172.21.15.96:0/2124820488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 msgr2=0x7fba00116f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 --2- 172.21.15.96:0/2124820488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba00116f30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009ca0 tx=0x7fb9f802a950 comp rx=0 tx=0).stop 2024-04-25T17:10:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 -- 172.21.15.96:0/2124820488 shutdown_connections 2024-04-25T17:10:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 --2- 172.21.15.96:0/2124820488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba00116f30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 --2- 172.21.15.96:0/2124820488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba001121b0 0x7fba001145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.343 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 --2- 172.21.15.96:0/2124820488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba00076470 0x7fba00076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 -- 172.21.15.96:0/2124820488 >> 172.21.15.96:0/2124820488 conn(0x7fba000ff230 msgr2=0x7fba00101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 -- 172.21.15.96:0/2124820488 shutdown_connections 2024-04-25T17:10:43.344 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.343+0000 7fba07620640 1 -- 172.21.15.96:0/2124820488 wait complete. 2024-04-25T17:10:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 Processor -- start 2024-04-25T17:10:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 -- start start 2024-04-25T17:10:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00076470 0x7fba001a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba001121b0 0x7fba001a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba001a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba0011a2f0 con 0x7fba00114b10 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba0011a170 con 0x7fba001121b0 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba07620640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba0011a470 con 0x7fba00076470 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba001a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba001a5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34874/0 (socket says 172.21.15.96:34874) 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 -- 172.21.15.96:0/327728519 learned_addr learned my addr 172.21.15.96:0/327728519 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba04bbb640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba001121b0 0x7fba001a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba053bc640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00076470 0x7fba001a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 -- 172.21.15.96:0/327728519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00076470 msgr2=0x7fba001a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00076470 0x7fba001a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 -- 172.21.15.96:0/327728519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba001121b0 msgr2=0x7fba001a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba001121b0 0x7fba001a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 -- 172.21.15.96:0/327728519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001a5b50 con 0x7fba00114b10 2024-04-25T17:10:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba053bc640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00076470 0x7fba001a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.344+0000 7fba05bbd640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba001a5590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8006010 tx=0x7fb9f802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.345+0000 7fb9f67fc640 1 -- 172.21.15.96:0/327728519 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8034040 con 0x7fba00114b10 2024-04-25T17:10:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.345+0000 7fb9f67fc640 1 -- 172.21.15.96:0/327728519 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f80308f0 con 0x7fba00114b10 2024-04-25T17:10:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.345+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba001a5de0 con 0x7fba00114b10 2024-04-25T17:10:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.345+0000 7fb9f67fc640 1 -- 172.21.15.96:0/327728519 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80047f0 con 0x7fba00114b10 2024-04-25T17:10:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.345+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a62d0 con 0x7fba00114b10 2024-04-25T17:10:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.347+0000 7fba0e5ae740 1 Processor -- start 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.347+0000 7fb9f67fc640 1 -- 172.21.15.96:0/327728519 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7fb9f8037020 con 0x7fba00114b10 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba0e5ae740 1 -- start start 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b73af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b74030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b74570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b47ada460 con 0x561b477d41b0 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b47a94590 con 0x561b477ab2f0 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561b47ad6420 con 0x561b477b34d0 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b73af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b73af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34888/0 (socket says 172.21.15.96:34888) 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 -- 172.21.15.96:0/1844962568 learned_addr learned my addr 172.21.15.96:0/1844962568 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fb9f67fc640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb9ec0afce0 0x7fb9ec0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba053bc640 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b74030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 -- 172.21.15.96:0/1844962568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 msgr2=0x561b47b74570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b74570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 -- 172.21.15.96:0/1844962568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 msgr2=0x561b47b74030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b74030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba05bbd640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb9ec0afce0 0x7fb9ec0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba04bbb640 1 -- 172.21.15.96:0/1844962568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b47aa10d0 con 0x561b477d41b0 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.348+0000 7fba053bc640 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b74030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.349+0000 7fb9f67fc640 1 -- 172.21.15.96:0/327728519 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7fb9f81000b0 con 0x7fba00114b10 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.349+0000 7fba04bbb640 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b73af0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000bcc0 tx=0x7fb9f00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6784e793eeed5f36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.349+0000 7fb9eb7fe640 1 -- 172.21.15.96:0/1844962568 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000be70 con 0x561b477d41b0 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.349+0000 7fb9eb7fe640 1 -- 172.21.15.96:0/1844962568 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f0016070 con 0x561b477d41b0 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.349+0000 7fb9eb7fe640 1 -- 172.21.15.96:0/1844962568 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00056c0 con 0x561b477d41b0 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 -- 172.21.15.96:0/1844962568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 msgr2=0x561b47b73af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba05bbd640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb9ec0afce0 0x7fb9ec0b21f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80fd8c0 tx=0x7fb9f80929d0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b73af0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000bcc0 tx=0x7fb9f00127c0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 -- 172.21.15.96:0/1844962568 shutdown_connections 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b74570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b74030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/1844962568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b73af0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 -- 172.21.15.96:0/1844962568 >> 172.21.15.96:0/1844962568 conn(0x561b477c8300 msgr2=0x561b47b702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 -- 172.21.15.96:0/1844962568 shutdown_connections 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 -- 172.21.15.96:0/1844962568 wait complete. 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 -- start start 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.350+0000 7fba0e5ae740 1 Processor -- start 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477ab2f0 0x561b47b7e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477b34d0 0x561b47b7e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b84170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b47b80c20 con 0x561b477d41b0 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b47a61b60 con 0x561b477b34d0 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561b47b80da0 con 0x561b477ab2f0 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba0e5ae740 10 client.0 _ll_register_callbacks cb 0x561b47a862b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b84170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b84170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34890/0 (socket says 172.21.15.96:34890) 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 -- 172.21.15.96:0/2479375163 learned_addr learned my addr 172.21.15.96:0/2479375163 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba053bc640 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477b34d0 0x561b47b7e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477ab2f0 0x561b47b7e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 -- 172.21.15.96:0/2479375163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477ab2f0 msgr2=0x561b47b7e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477ab2f0 0x561b47b7e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 -- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477b34d0 msgr2=0x561b47b7e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477b34d0 0x561b47b7e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 -- 172.21.15.96:0/2479375163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b477ec210 con 0x561b477d41b0 2024-04-25T17:10:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fba04bbb640 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b84170 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000bad0 tx=0x7fb9f0005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77ab600b49d96a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fb9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.351+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/2479375163 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0004220 con 0x561b477d41b0 2024-04-25T17:10:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/2479375163 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7fb9f00043c0 con 0x561b477d41b0 2024-04-25T17:10:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f0016070 con 0x561b477d41b0 2024-04-25T17:10:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/2479375163 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb9f00048d0 con 0x561b477d41b0 2024-04-25T17:10:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 10 client.15378 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x561b478b8620 con 0x561b477d41b0 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/2479375163 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f0016070 con 0x561b477d41b0 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/2479375163 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fb9f0003c50 con 0x561b477d41b0 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 10 client.15378 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 20 client.15378 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b47792d80 0x561b47b906c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 4 client.15378 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.352+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b47ae8070 con 0x561b47792d80 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.353+0000 7fba053bc640 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b47792d80 0x561b47b906c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.353+0000 7fb9e8ff9640 10 client.15378 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.353+0000 7fba053bc640 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b47792d80 0x561b47b906c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c789ec5b41eff28 server_cookie=64700dc7df72c16f in_seq=0 out_seq=0 2024-04-25T17:10:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/2479375163 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561b47ae8070 con 0x561b47792d80 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fb9e8ff9640 10 client.15378 handle_command_reply: tid=0 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 client.15378 shutdown 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 msgr2=0x561b47b84170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b84170 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000bad0 tx=0x7fb9f0005340 comp rx=0 tx=0).stop 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 shutdown_connections 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b47792d80 0x561b47b906c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47b84170 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477b34d0 0x561b47b7e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/2479375163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477ab2f0 0x561b47b7e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fb9e8ff9640 0 client.15378 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 >> 172.21.15.96:0/2479375163 conn(0x561b477c8300 msgr2=0x561b47b73920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 shutdown_connections 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 1 -- 172.21.15.96:0/2479375163 wait complete. 2024-04-25T17:10:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.354+0000 7fba0e5ae740 20 client.15378 trim_cache size 0 max 16384 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 Processor -- start 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 -- start start 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477ab2f0 0x561b47b77e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b8a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477d41b0 0x561b47b8a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b3ec0 con 0x561b477ab2f0 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b6740 con 0x561b477d41b0 2024-04-25T17:10:43.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.369+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b4530 con 0x561b477b34d0 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477ab2f0 0x561b47b77e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477ab2f0 0x561b47b77e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34896/0 (socket says 172.21.15.96:34896) 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 -- 172.21.15.96:0/3370087926 learned_addr learned my addr 172.21.15.96:0/3370087926 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba053bc640 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b8a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 -- 172.21.15.96:0/3370087926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 msgr2=0x561b47b8a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b8a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 -- 172.21.15.96:0/3370087926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477d41b0 msgr2=0x561b47b8a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477d41b0 0x561b47b8a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 -- 172.21.15.96:0/3370087926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b47b7eeb0 con 0x561b477ab2f0 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fba04bbb640 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477ab2f0 0x561b47b77e50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000b470 tx=0x7fb9f0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489351285d1e3e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fb9eaffd640 1 -- 172.21.15.96:0/3370087926 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000b930 con 0x561b477ab2f0 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fb9eaffd640 1 -- 172.21.15.96:0/3370087926 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f0016040 con 0x561b477ab2f0 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.370+0000 7fb9eaffd640 1 -- 172.21.15.96:0/3370087926 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00043a0 con 0x561b477ab2f0 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 -- 172.21.15.96:0/3370087926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477ab2f0 msgr2=0x561b47b77e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477ab2f0 0x561b47b77e50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000b470 tx=0x7fb9f0005270 comp rx=0 tx=0).stop 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 -- 172.21.15.96:0/3370087926 shutdown_connections 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477d41b0 0x561b47b8a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b8a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/3370087926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477ab2f0 0x561b47b77e50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 -- 172.21.15.96:0/3370087926 >> 172.21.15.96:0/3370087926 conn(0x561b477c8300 msgr2=0x561b47b77740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 -- 172.21.15.96:0/3370087926 shutdown_connections 2024-04-25T17:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 -- 172.21.15.96:0/3370087926 wait complete. 2024-04-25T17:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 -- start start 2024-04-25T17:10:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.371+0000 7fba0e5ae740 1 Processor -- start 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b89630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b89b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba0e5ae740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47bf07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b4c00 con 0x561b477d41b0 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b6070 con 0x561b477ab2f0 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba0e5ae740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b52d0 con 0x561b477b34d0 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba0e5ae740 10 client.0 _ll_register_callbacks cb 0x7fb9f000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47bf07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47bf07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34900/0 (socket says 172.21.15.96:34900) 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 -- 172.21.15.96:0/4045870640 learned_addr learned my addr 172.21.15.96:0/4045870640 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b89630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba053bc640 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b89b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 -- 172.21.15.96:0/4045870640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 msgr2=0x561b47b89b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b89b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 -- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 msgr2=0x561b47b89630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b89630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 -- 172.21.15.96:0/4045870640 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b47bf0d30 con 0x561b477d41b0 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fba04bbb640 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47bf07c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0012190 tx=0x7fb9f0012340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1a55d2178b0c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fb9e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.372+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/4045870640 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f003b040 con 0x561b477d41b0 2024-04-25T17:10:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/4045870640 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7fb9f0004a70 con 0x561b477d41b0 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f0016040 con 0x561b477d41b0 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/4045870640 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fb9f0004c70 con 0x561b477d41b0 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 10 client.15384 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x561b47bf8040 con 0x561b477d41b0 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/4045870640 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f0016040 con 0x561b477d41b0 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/4045870640 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fb9f0041070 con 0x561b477d41b0 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 10 client.15384 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 20 client.15384 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b477681c0 0x561b47bfa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 4 client.15384 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b47ae1ee0 con 0x561b477681c0 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.373+0000 7fba053bc640 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b477681c0 0x561b47bfa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.374+0000 7fb9e8ff9640 10 client.15384 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.374+0000 7fba053bc640 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b477681c0 0x561b47bfa5b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4de19108f1cc31 server_cookie=938992620baed935 in_seq=0 out_seq=0 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fb9e8ff9640 1 -- 172.21.15.96:0/4045870640 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b47ae1ee0 con 0x561b477681c0 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fb9e8ff9640 10 client.15384 handle_command_reply: tid=0 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 client.15384 shutdown 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 msgr2=0x561b47bf07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47bf07c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0012190 tx=0x7fb9f0012340 comp rx=0 tx=0).stop 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 shutdown_connections 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x561b477681c0 0x561b47bfa5b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b477d41b0 0x561b47bf07c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561b477b34d0 0x561b47b89b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fb9e8ff9640 0 client.15384 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 --2- 172.21.15.96:0/4045870640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b477ab2f0 0x561b47b89630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 >> 172.21.15.96:0/4045870640 conn(0x561b477c8300 msgr2=0x561b47b73880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 shutdown_connections 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 1 -- 172.21.15.96:0/4045870640 wait complete. 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba0e5ae740 20 client.15384 trim_cache size 0 max 16384 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:10:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:10:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:10:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:10:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb9ec0afce0 msgr2=0x7fb9ec0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba07620640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb9ec0afce0 0x7fb9ec0b21f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80fd8c0 tx=0x7fb9f80929d0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 msgr2=0x7fba001a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.378+0000 7fba07620640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba001a5590 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8006010 tx=0x7fb9f802f5d0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 shutdown_connections 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba00114b10 0x7fba001a5590 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb9ec0afce0 0x7fb9ec0b21f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba001121b0 0x7fba001a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 --2- 172.21.15.96:0/327728519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00076470 0x7fba001a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 >> 172.21.15.96:0/327728519 conn(0x7fba000ff230 msgr2=0x7fba00111a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 shutdown_connections 2024-04-25T17:10:43.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:43.379+0000 7fba07620640 1 -- 172.21.15.96:0/327728519 wait complete. 2024-04-25T17:10:44.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:44 smithi096 ceph-mon[29131]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T17:10:44.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:44 smithi097 ceph-mon[32602]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T17:10:44.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:44 smithi171 ceph-mon[33189]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T17:10:46.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:46 smithi096 ceph-mon[29131]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:46.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:46 smithi097 ceph-mon[32602]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:46.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:46 smithi171 ceph-mon[33189]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:10:48.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:48 smithi096 ceph-mon[29131]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:48.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:48 smithi097 ceph-mon[32602]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:48.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:48 smithi171 ceph-mon[33189]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:50.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:50 smithi096 ceph-mon[29131]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:50.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:50 smithi097 ceph-mon[32602]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:50.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:50 smithi171 ceph-mon[33189]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:52.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:52 smithi096 ceph-mon[29131]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:52.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:52 smithi097 ceph-mon[32602]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:52 smithi171 ceph-mon[33189]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:54.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:54 smithi096 ceph-mon[29131]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:54.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:54 smithi097 ceph-mon[32602]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:54.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:54 smithi171 ceph-mon[33189]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 -- 172.21.15.96:0/1522657996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4108ca0 msgr2=0x7f99c4102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 --2- 172.21.15.96:0/1522657996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4108ca0 0x7f99c4102d60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009ca0 tx=0x7f99b402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 -- 172.21.15.96:0/1522657996 shutdown_connections 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 --2- 172.21.15.96:0/1522657996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c4103910 0x7f99c410f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 --2- 172.21.15.96:0/1522657996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99c4109140 0x7f99c41032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 --2- 172.21.15.96:0/1522657996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4108ca0 0x7f99c4102d60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 -- 172.21.15.96:0/1522657996 >> 172.21.15.96:0/1522657996 conn(0x7f99c40fd8a0 msgr2=0x7f99c40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 -- 172.21.15.96:0/1522657996 shutdown_connections 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 -- 172.21.15.96:0/1522657996 wait complete. 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.488+0000 7f99c9813640 1 Processor -- start 2024-04-25T17:10:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c9813640 1 -- start start 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c9813640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4103910 0x7f99c41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c9813640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c4108ca0 0x7f99c41a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c9813640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99c4109140 0x7f99c419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c9813640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4112e90 con 0x7f99c4103910 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c9813640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4112d10 con 0x7f99c4109140 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c9813640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4113010 con 0x7f99c4108ca0 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4103910 0x7f99c41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4103910 0x7f99c41a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34600/0 (socket says 172.21.15.96:34600) 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 -- 172.21.15.96:0/700817948 learned_addr learned my addr 172.21.15.96:0/700817948 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c37fe640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99c4109140 0x7f99c419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c27fc640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c4108ca0 0x7f99c41a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 -- 172.21.15.96:0/700817948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c4108ca0 msgr2=0x7f99c41a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c4108ca0 0x7f99c41a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 -- 172.21.15.96:0/700817948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99c4109140 msgr2=0x7f99c419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99c4109140 0x7f99c419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c2ffd640 1 -- 172.21.15.96:0/700817948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c419aee0 con 0x7f99c4103910 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.489+0000 7f99c37fe640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99c4109140 0x7f99c419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.490+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4103910 0x7f99c41a05d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006010 tx=0x7f99b40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.490+0000 7f99c8811640 1 -- 172.21.15.96:0/700817948 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4003bf0 con 0x7f99c4103910 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.490+0000 7f99c8811640 1 -- 172.21.15.96:0/700817948 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b4003d90 con 0x7f99c4103910 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.490+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c419b170 con 0x7f99c4103910 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.490+0000 7f99c8811640 1 -- 172.21.15.96:0/700817948 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b402fb60 con 0x7f99c4103910 2024-04-25T17:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.490+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c41a7420 con 0x7f99c4103910 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.492+0000 7f99d079d740 1 Processor -- start 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c8811640 1 -- 172.21.15.96:0/700817948 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 147696+0+0 (secure 0 0 0) 0x7f99b402fd00 con 0x7f99c4103910 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99d079d740 1 -- start start 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99d079d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74334d1b0 0x55a7436ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99d079d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99d079d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74332c4d0 0x55a7436ed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99d079d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a743653460 con 0x55a74332c4d0 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99d079d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a74360d590 con 0x55a74334d1b0 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99d079d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a74364f420 con 0x55a7433242f0 2024-04-25T17:10:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74332c4d0 0x55a7436ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74332c4d0 0x55a7436ed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34612/0 (socket says 172.21.15.96:34612) 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 -- 172.21.15.96:0/265572901 learned_addr learned my addr 172.21.15.96:0/265572901 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c37fe640 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74334d1b0 0x55a7436ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c8811640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f99ac0afce0 0x7f99ac0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 -- 172.21.15.96:0/265572901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 msgr2=0x55a7436ed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436ed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 -- 172.21.15.96:0/265572901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74334d1b0 msgr2=0x55a7436ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74334d1b0 0x55a7436ecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c27fc640 1 -- 172.21.15.96:0/265572901 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74361a0d0 con 0x55a74332c4d0 2024-04-25T17:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.493+0000 7f99c37fe640 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74334d1b0 0x55a7436ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.494+0000 7f99c37fe640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f99ac0afce0 0x7f99ac0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.494+0000 7f99c8811640 1 -- 172.21.15.96:0/700817948 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f99b4100660 con 0x7f99c4103910 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.494+0000 7f99c27fc640 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74332c4d0 0x55a7436ed570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f99b800bb70 tx=0x7f99b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1fc40b2c840122 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.494+0000 7f99a9ffb640 1 -- 172.21.15.96:0/265572901 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004030 con 0x55a74332c4d0 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.494+0000 7f99a9ffb640 1 -- 172.21.15.96:0/265572901 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b8005020 con 0x55a74332c4d0 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.494+0000 7f99a9ffb640 1 -- 172.21.15.96:0/265572901 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004ba0 con 0x55a74332c4d0 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 -- 172.21.15.96:0/265572901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74332c4d0 msgr2=0x55a7436ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99c37fe640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f99ac0afce0 0x7f99ac0b21f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x55a7436e9c60 tx=0x7f99bc0046c0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74332c4d0 0x55a7436ed570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f99b800bb70 tx=0x7f99b80107a0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 -- 172.21.15.96:0/265572901 shutdown_connections 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74332c4d0 0x55a7436ed570 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 --2- 172.21.15.96:0/265572901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74334d1b0 0x55a7436ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 -- 172.21.15.96:0/265572901 >> 172.21.15.96:0/265572901 conn(0x55a743341300 msgr2=0x55a7436e92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 -- 172.21.15.96:0/265572901 shutdown_connections 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 -- 172.21.15.96:0/265572901 wait complete. 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.495+0000 7f99d079d740 1 -- start start 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 1 Processor -- start 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436f7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74334d1b0 0x55a7436fd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a7436f9c20 con 0x55a74334d1b0 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a7435dab60 con 0x55a74332c4d0 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a7436f9da0 con 0x55a7433242f0 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99d079d740 10 client.0 _ll_register_callbacks cb 0x55a7435ff2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74334d1b0 0x55a7436fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74334d1b0 0x55a7436fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34618/0 (socket says 172.21.15.96:34618) 2024-04-25T17:10:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 -- 172.21.15.96:0/1869001066 learned_addr learned my addr 172.21.15.96:0/1869001066 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c27fc640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 -- 172.21.15.96:0/1869001066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 msgr2=0x55a7436f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 -- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 msgr2=0x55a7436f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 -- 172.21.15.96:0/1869001066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a743365210 con 0x55a74334d1b0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99c37fe640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74334d1b0 0x55a7436fd110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f99bc012620 tx=0x7f99bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8160ef6300872bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.496+0000 7f99a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99a2ffd640 1 -- 172.21.15.96:0/1869001066 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc01d930 con 0x55a74334d1b0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99a2ffd640 1 -- 172.21.15.96:0/1869001066 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f99bc01dad0 con 0x55a74334d1b0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b8005020 con 0x55a74334d1b0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99a2ffd640 1 -- 172.21.15.96:0/1869001066 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f99bc01dcd0 con 0x55a74334d1b0 2024-04-25T17:10:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 10 client.15396 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55a743431620 con 0x55a74334d1b0 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99a2ffd640 1 -- 172.21.15.96:0/1869001066 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b8005020 con 0x55a74334d1b0 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99a2ffd640 1 -- 172.21.15.96:0/1869001066 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f99bc027500 con 0x55a74334d1b0 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 10 client.15396 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 10 client.15396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 20 client.15396 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a74330bd80 0x55a743709740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 4 client.15396 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.497+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a743661070 con 0x55a74330bd80 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.498+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a74330bd80 0x55a743709740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.498+0000 7f99a2ffd640 10 client.15396 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.498+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a74330bd80 0x55a743709740 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c532fa7e1a58c server_cookie=a402adbbc7afe01f in_seq=0 out_seq=0 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99a2ffd640 1 -- 172.21.15.96:0/1869001066 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a743661070 con 0x55a74330bd80 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99a2ffd640 10 client.15396 handle_command_reply: tid=0 2024-04-25T17:10:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 client.15396 shutdown 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74334d1b0 msgr2=0x55a7436fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74334d1b0 0x55a7436fd110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f99bc012620 tx=0x7f99bc004730 comp rx=0 tx=0).stop 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 shutdown_connections 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a74330bd80 0x55a743709740 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74334d1b0 0x55a7436fd110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99a2ffd640 0 client.15396 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 --2- 172.21.15.96:0/1869001066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7433242f0 0x55a7436f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 >> 172.21.15.96:0/1869001066 conn(0x55a743341300 msgr2=0x55a7436ee810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 shutdown_connections 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 1 -- 172.21.15.96:0/1869001066 wait complete. 2024-04-25T17:10:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.499+0000 7f99d079d740 20 client.15396 trim_cache size 0 max 16384 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 Processor -- start 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 -- start start 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a7436eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 0x55a7436f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41ada00 con 0x55a7433242f0 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b0280 con 0x55a74332c4d0 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99d079d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41ae070 con 0x55a74334d1b0 2024-04-25T17:10:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a7436eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a7436eda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34632/0 (socket says 172.21.15.96:34632) 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.514+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3128058897 learned_addr learned my addr 172.21.15.96:0/3128058897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3128058897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 msgr2=0x55a7436f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/3128058897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 0x55a7436f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3128058897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 msgr2=0x55a7436f07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/3128058897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3128058897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7436f2b40 con 0x55a7433242f0 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/3128058897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a7436eda60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99b40c4830 tx=0x7f99b4102be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bcde0c0c3910f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99a97fa640 1 -- 172.21.15.96:0/3128058897 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b40c8100 con 0x55a7433242f0 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99a97fa640 1 -- 172.21.15.96:0/3128058897 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b4036040 con 0x55a7433242f0 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.515+0000 7f99a97fa640 1 -- 172.21.15.96:0/3128058897 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b40c7f20 con 0x55a7433242f0 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 -- 172.21.15.96:0/3128058897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 msgr2=0x55a7436eda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 --2- 172.21.15.96:0/3128058897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a7436eda60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99b40c4830 tx=0x7f99b4102be0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 -- 172.21.15.96:0/3128058897 shutdown_connections 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 --2- 172.21.15.96:0/3128058897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 0x55a7436f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 --2- 172.21.15.96:0/3128058897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 --2- 172.21.15.96:0/3128058897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a7436eda60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 -- 172.21.15.96:0/3128058897 >> 172.21.15.96:0/3128058897 conn(0x55a743341300 msgr2=0x55a7436f5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 -- 172.21.15.96:0/3128058897 shutdown_connections 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 -- 172.21.15.96:0/3128058897 wait complete. 2024-04-25T17:10:56.517 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 -- start start 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.516+0000 7f99d079d740 1 Processor -- start 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a743703360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 0x55a7436febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41ae740 con 0x55a7433242f0 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41afbb0 con 0x55a74332c4d0 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41aee10 con 0x55a74334d1b0 2024-04-25T17:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 10 client.0 _ll_register_callbacks cb 0x7f99bc00ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a743703360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a743703360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34646/0 (socket says 172.21.15.96:34646) 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3056917205 learned_addr learned my addr 172.21.15.96:0/3056917205 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c27fc640 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3056917205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 msgr2=0x55a7436febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c37fe640 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 0x55a7436febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 0x55a7436febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 msgr2=0x55a7436fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436fe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 -- 172.21.15.96:0/3056917205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7436ff130 con 0x55a7433242f0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a743703360 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f99b4005e30 tx=0x7f99b402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955ee68abd6a29f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99a2ffd640 1 -- 172.21.15.96:0/3056917205 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b40c7f20 con 0x55a7433242f0 2024-04-25T17:10:56.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99a2ffd640 1 -- 172.21.15.96:0/3056917205 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f99b40c4da0 con 0x55a7433242f0 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.517+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b4036040 con 0x55a7433242f0 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99a2ffd640 1 -- 172.21.15.96:0/3056917205 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f99b40c8100 con 0x55a7433242f0 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 10 client.15402 fetch_fsmap learned FSMap version 18 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55a7437716a0 con 0x55a7433242f0 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99a2ffd640 1 -- 172.21.15.96:0/3056917205 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b4036040 con 0x55a7433242f0 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99a2ffd640 1 -- 172.21.15.96:0/3056917205 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f99b40b10c0 con 0x55a7433242f0 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 10 client.15402 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 10 client.15402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 20 client.15402 populate_metadata read hostname 'smithi096' 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a7432e11c0 0x55a743773da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 4 client.15402 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a74365aee0 con 0x55a7432e11c0 2024-04-25T17:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.518+0000 7f99c27fc640 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a7432e11c0 0x55a743773da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:10:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.519+0000 7f99a2ffd640 10 client.15402 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.519+0000 7f99c27fc640 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a7432e11c0 0x55a743773da0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4950433d43c4765 server_cookie=52d3ccef5319274a in_seq=0 out_seq=0 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.526+0000 7f99a2ffd640 1 -- 172.21.15.96:0/3056917205 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a74365aee0 con 0x55a7432e11c0 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.526+0000 7f99a2ffd640 10 client.15402 handle_command_reply: tid=0 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.526+0000 7f99d079d740 1 client.15402 shutdown 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 msgr2=0x55a743703360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a743703360 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f99b4005e30 tx=0x7f99b402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 shutdown_connections 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55a7432e11c0 0x55a743773da0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a74334d1b0 0x55a7436febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99a2ffd640 0 client.15402 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a74332c4d0 0x55a7436fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 --2- 172.21.15.96:0/3056917205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a7433242f0 0x55a743703360 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 >> 172.21.15.96:0/3056917205 conn(0x55a743341300 msgr2=0x55a7436ee730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 shutdown_connections 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 1 -- 172.21.15.96:0/3056917205 wait complete. 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99d079d740 20 client.15402 trim_cache size 0 max 16384 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:10:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f99ac0afce0 msgr2=0x7f99ac0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f99ac0afce0 0x7f99ac0b21f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x55a7436e9c60 tx=0x7f99bc0046c0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4103910 msgr2=0x7f99c41a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:10:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4103910 0x7f99c41a05d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f99b4006010 tx=0x7f99b40319f0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 shutdown_connections 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f99ac0afce0 0x7f99ac0b21f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99c4109140 0x7f99c419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c4108ca0 0x7f99c41a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 --2- 172.21.15.96:0/700817948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4103910 0x7f99c41a05d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.527+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 >> 172.21.15.96:0/700817948 conn(0x7f99c40fd8a0 msgr2=0x7f99c40fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.528+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 shutdown_connections 2024-04-25T17:10:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:10:56.528+0000 7f99c9813640 1 -- 172.21.15.96:0/700817948 wait complete. 2024-04-25T17:10:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:56 smithi096 ceph-mon[29131]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:56.782 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:56 smithi171 ceph-mon[33189]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:56.938 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:56 smithi097 ceph-mon[32602]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:10:58.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:10:58 smithi096 ceph-mon[29131]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-25T17:10:58.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:10:58 smithi097 ceph-mon[32602]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-25T17:10:58.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:10:58 smithi171 ceph-mon[33189]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-25T17:11:00.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:00 smithi096 ceph-mon[29131]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-04-25T17:11:00.689 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:11:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:11:00.223+0000 7f469c323700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:11:00.689 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:11:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:11:00.223+0000 7f469c323700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:11:00.689 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:11:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:11:00.223+0000 7f469c323700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:11:00.689 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:11:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:11:00.223+0000 7f469c323700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:11:00.689 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:11:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:11:00.223+0000 7f469c323700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T17:11:00.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:00 smithi097 ceph-mon[32602]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-04-25T17:11:00.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:00 smithi171 ceph-mon[33189]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-04-25T17:11:02.581 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:02 smithi171 ceph-mon[33189]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:02.582 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:02 smithi171 ceph-mon[33189]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-04-25T17:11:02.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:02 smithi096 ceph-mon[29131]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:02.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:02 smithi096 ceph-mon[29131]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-04-25T17:11:02.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:02 smithi097 ceph-mon[32602]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:02.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:02 smithi097 ceph-mon[32602]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-04-25T17:11:04.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:04 smithi096 ceph-mon[29131]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:04.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:04 smithi097 ceph-mon[32602]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:04.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:04 smithi171 ceph-mon[33189]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:06.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:06 smithi096 ceph-mon[29131]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T17:11:06.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:06 smithi097 ceph-mon[32602]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T17:11:06.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:06 smithi171 ceph-mon[33189]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-04-25T17:11:08.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:08 smithi096 ceph-mon[29131]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:08.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:08 smithi097 ceph-mon[32602]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:08.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:08 smithi171 ceph-mon[33189]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 -- 172.21.15.96:0/3350460158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 msgr2=0x7f04641050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 --2- 172.21.15.96:0/3350460158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f04641050a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0460009900 tx=0x7f046002a410 comp rx=0 tx=0).stop 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 -- 172.21.15.96:0/3350460158 shutdown_connections 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 --2- 172.21.15.96:0/3350460158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f046410e0e0 0x7f04641105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 --2- 172.21.15.96:0/3350460158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f046410b6f0 0x7f046410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 --2- 172.21.15.96:0/3350460158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f04641050a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 -- 172.21.15.96:0/3350460158 >> 172.21.15.96:0/3350460158 conn(0x7f04640fe260 msgr2=0x7f0464100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 -- 172.21.15.96:0/3350460158 shutdown_connections 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.635+0000 7f046aabf640 1 -- 172.21.15.96:0/3350460158 wait complete. 2024-04-25T17:11:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 Processor -- start 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 -- start start 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f046419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f046410b6f0 0x7f046419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f046410e0e0 0x7f046419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0464113d90 con 0x7f0464104c90 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0464113c10 con 0x7f046410b6f0 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.636+0000 7f046aabf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0464113f10 con 0x7f046410e0e0 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f046419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f046419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54564/0 (socket says 172.21.15.96:54564) 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 -- 172.21.15.96:0/3979570897 learned_addr learned my addr 172.21.15.96:0/3979570897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046905c640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f046410e0e0 0x7f046419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f045bfff640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f046410b6f0 0x7f046419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 -- 172.21.15.96:0/3979570897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f046410e0e0 msgr2=0x7f046419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f046410e0e0 0x7f046419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 -- 172.21.15.96:0/3979570897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f046410b6f0 msgr2=0x7f046419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f046410b6f0 0x7f046419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 -- 172.21.15.96:0/3979570897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046419f530 con 0x7f0464104c90 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046905c640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f046410e0e0 0x7f046419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046885b640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f046419a750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f046002aa00 tx=0x7f0460003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f0459ffb640 1 -- 172.21.15.96:0/3979570897 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046003f070 con 0x7f0464104c90 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f0459ffb640 1 -- 172.21.15.96:0/3979570897 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0460003ec0 con 0x7f0464104c90 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046419f7c0 con 0x7f0464104c90 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.637+0000 7f0459ffb640 1 -- 172.21.15.96:0/3979570897 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460035030 con 0x7f0464104c90 2024-04-25T17:11:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.638+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046419fd00 con 0x7f0464104c90 2024-04-25T17:11:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 Processor -- start 2024-04-25T17:11:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0459ffb640 1 -- 172.21.15.96:0/3979570897 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f0460042070 con 0x7f0464104c90 2024-04-25T17:11:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 -- start start 2024-04-25T17:11:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968bdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964f52f0 0x5556968be030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964fd4d0 0x5556968be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555696824460 con 0x5556964fd4d0 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5556967de590 con 0x55569651e1b0 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f0471970740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555696820420 con 0x5556964f52f0 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.640+0000 7f045bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964fd4d0 0x5556968be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964fd4d0 0x5556968be570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54568/0 (socket says 172.21.15.96:54568) 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 -- 172.21.15.96:0/3554207456 learned_addr learned my addr 172.21.15.96:0/3554207456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f046885b640 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964f52f0 0x5556968be030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f0459ffb640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f044c0afdf0 0x7f044c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f046905c640 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968bdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f046905c640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f044c0afdf0 0x7f044c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 -- 172.21.15.96:0/3554207456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964f52f0 msgr2=0x5556968be030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964f52f0 0x5556968be030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 -- 172.21.15.96:0/3554207456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 msgr2=0x5556968bdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968bdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 -- 172.21.15.96:0/3554207456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556967eb0d0 con 0x5556964fd4d0 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f0459ffb640 1 -- 172.21.15.96:0/3979570897 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f04600ffad0 con 0x7f0464104c90 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f045bfff640 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964fd4d0 0x5556968be570 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f045400bc90 tx=0x7f045400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fbe7ece0800f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f046905c640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f044c0afdf0 0x7f044c0b2300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f045c007fd0 tx=0x7f045c0072a0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f044affd640 1 -- 172.21.15.96:0/3554207456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454015070 con 0x5556964fd4d0 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f044affd640 1 -- 172.21.15.96:0/3554207456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045401a910 con 0x5556964fd4d0 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.641+0000 7f044affd640 1 -- 172.21.15.96:0/3554207456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454011860 con 0x5556964fd4d0 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 -- 172.21.15.96:0/3554207456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964fd4d0 msgr2=0x5556968be570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964fd4d0 0x5556968be570 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f045400bc90 tx=0x7f045400f730 comp rx=0 tx=0).stop 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 -- 172.21.15.96:0/3554207456 shutdown_connections 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964fd4d0 0x5556968be570 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964f52f0 0x5556968be030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 --2- 172.21.15.96:0/3554207456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968bdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 -- 172.21.15.96:0/3554207456 >> 172.21.15.96:0/3554207456 conn(0x555696512300 msgr2=0x5556968ba2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 -- 172.21.15.96:0/3554207456 shutdown_connections 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.642+0000 7f0471970740 1 -- 172.21.15.96:0/3554207456 wait complete. 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 -- start start 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 Processor -- start 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 0x5556968c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5556968cac20 con 0x5556964f52f0 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5556967abb60 con 0x55569651e1b0 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5556968cada0 con 0x5556964fd4d0 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f0471970740 10 client.0 _ll_register_callbacks cb 0x5556967d02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54578/0 (socket says 172.21.15.96:54578) 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.643+0000 7f046885b640 1 -- 172.21.15.96:0/4115540948 learned_addr learned my addr 172.21.15.96:0/4115540948 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046905c640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f045bfff640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 0x5556968c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046885b640 1 -- 172.21.15.96:0/4115540948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 msgr2=0x5556968c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046885b640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 0x5556968c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046885b640 1 -- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 msgr2=0x5556968ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046885b640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046885b640 1 -- 172.21.15.96:0/4115540948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555696536210 con 0x5556964f52f0 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046905c640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f046885b640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968c81e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04600c7800 tx=0x7f046003b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2105fa599ad6cf8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0443fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:11:09.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0443fff640 1 -- 172.21.15.96:0/4115540948 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460049070 con 0x5556964f52f0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0443fff640 1 -- 172.21.15.96:0/4115540948 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f04600d06e0 con 0x5556964f52f0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045401a910 con 0x5556964f52f0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0443fff640 1 -- 172.21.15.96:0/4115540948 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f04600d02a0 con 0x5556964f52f0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0471970740 10 client.15420 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x555696602620 con 0x5556964f52f0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0443fff640 1 -- 172.21.15.96:0/4115540948 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045401a910 con 0x5556964f52f0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0443fff640 1 -- 172.21.15.96:0/4115540948 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f04600c7d60 con 0x5556964f52f0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.644+0000 7f0471970740 10 client.15420 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f0471970740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f0471970740 20 client.15420 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f0471970740 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964dcd80 0x5556968da790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f0471970740 4 client.15420 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555696832070 con 0x5556964dcd80 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f045bfff640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964dcd80 0x5556968da790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f045bfff640 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964dcd80 0x5556968da790 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c33c945071864 server_cookie=8d93bbaf85c16bff in_seq=0 out_seq=0 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.645+0000 7f0443fff640 10 client.15420 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0443fff640 1 -- 172.21.15.96:0/4115540948 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555696832070 con 0x5556964dcd80 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0443fff640 10 client.15420 handle_command_reply: tid=0 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 client.15420 shutdown 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 msgr2=0x5556968c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968c81e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04600c7800 tx=0x7f046003b890 comp rx=0 tx=0).stop 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 shutdown_connections 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964dcd80 0x5556968da790 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 0x5556968c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 --2- 172.21.15.96:0/4115540948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968c81e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 >> 172.21.15.96:0/4115540948 conn(0x555696512300 msgr2=0x5556968bf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0443fff640 0 client.15420 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 shutdown_connections 2024-04-25T17:11:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 1 -- 172.21.15.96:0/4115540948 wait complete. 2024-04-25T17:11:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.646+0000 7f0471970740 20 client.15420 trim_cache size 0 max 16384 2024-04-25T17:11:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.661+0000 7f0471970740 1 Processor -- start 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.661+0000 7f0471970740 1 -- start start 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f0471970740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f0471970740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 0x5556968be890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f0471970740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968dff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f0471970740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04641ad890 con 0x5556964f52f0 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f0471970740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04641b0110 con 0x55569651e1b0 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f0471970740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04641adf00 con 0x5556964fd4d0 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968bfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54586/0 (socket says 172.21.15.96:54586) 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 -- 172.21.15.96:0/3717627570 learned_addr learned my addr 172.21.15.96:0/3717627570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046905c640 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968dff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 -- 172.21.15.96:0/3717627570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 msgr2=0x5556968be890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 0x5556968be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 -- 172.21.15.96:0/3717627570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 msgr2=0x5556968dff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968dff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 -- 172.21.15.96:0/3717627570 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556968c8ec0 con 0x5556964f52f0 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f046885b640 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968bfeb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f04600c8160 tx=0x7f04600fd1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f36bb2e279c5f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f044a7fc640 1 -- 172.21.15.96:0/3717627570 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460049070 con 0x5556964f52f0 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f044a7fc640 1 -- 172.21.15.96:0/3717627570 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04600c8200 con 0x5556964f52f0 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.662+0000 7f044a7fc640 1 -- 172.21.15.96:0/3717627570 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600fd3e0 con 0x5556964f52f0 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 -- 172.21.15.96:0/3717627570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 msgr2=0x5556968bfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968bfeb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f04600c8160 tx=0x7f04600fd1c0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 -- 172.21.15.96:0/3717627570 shutdown_connections 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55569651e1b0 0x5556968dff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5556964fd4d0 0x5556968be890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 --2- 172.21.15.96:0/3717627570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968bfeb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 -- 172.21.15.96:0/3717627570 >> 172.21.15.96:0/3717627570 conn(0x555696512300 msgr2=0x5556968bf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 -- 172.21.15.96:0/3717627570 shutdown_connections 2024-04-25T17:11:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.663+0000 7f0471970740 1 -- 172.21.15.96:0/3717627570 wait complete. 2024-04-25T17:11:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 -- start start 2024-04-25T17:11:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 Processor -- start 2024-04-25T17:11:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556964fd4d0 0x5556968cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55569651e1b0 0x5556968cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04641ae5d0 con 0x5556964f52f0 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04641afa40 con 0x5556964fd4d0 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04641aeca0 con 0x55569651e1b0 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f0471970740 10 client.0 _ll_register_callbacks cb 0x7f0460005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046885b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54598/0 (socket says 172.21.15.96:54598) 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046885b640 1 -- 172.21.15.96:0/1591936690 learned_addr learned my addr 172.21.15.96:0/1591936690 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f045bfff640 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556964fd4d0 0x5556968cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046905c640 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55569651e1b0 0x5556968cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046885b640 1 -- 172.21.15.96:0/1591936690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55569651e1b0 msgr2=0x5556968cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046885b640 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55569651e1b0 0x5556968cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046885b640 1 -- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556964fd4d0 msgr2=0x5556968cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.664+0000 7f046885b640 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556964fd4d0 0x5556968cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f046885b640 1 -- 172.21.15.96:0/1591936690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555696940b60 con 0x5556964f52f0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f046885b640 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968cf130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0460005f70 tx=0x7f04600fd5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6787de5c797ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0443fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0443fff640 1 -- 172.21.15.96:0/1591936690 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460049070 con 0x5556964f52f0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0443fff640 1 -- 172.21.15.96:0/1591936690 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f04600c6f70 con 0x5556964f52f0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04600c8200 con 0x5556964f52f0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0443fff640 1 -- 172.21.15.96:0/1591936690 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f04600c39c0 con 0x5556964f52f0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 10 client.15426 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x555696940b60 con 0x5556964f52f0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0443fff640 1 -- 172.21.15.96:0/1591936690 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04600c8200 con 0x5556964f52f0 2024-04-25T17:11:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0443fff640 1 -- 172.21.15.96:0/1591936690 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f04600ea1d0 con 0x5556964f52f0 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 10 client.15426 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 20 client.15426 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964b21c0 0x555696944100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 4 client.15426 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.665+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55569682bee0 con 0x5556964b21c0 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.666+0000 7f045bfff640 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964b21c0 0x555696944100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.666+0000 7f045bfff640 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964b21c0 0x555696944100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce19d80ccd0e991 server_cookie=fc4553df65f97202 in_seq=0 out_seq=0 2024-04-25T17:11:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.666+0000 7f0443fff640 10 client.15426 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0443fff640 1 -- 172.21.15.96:0/1591936690 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55569682bee0 con 0x5556964b21c0 2024-04-25T17:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0443fff640 10 client.15426 handle_command_reply: tid=0 2024-04-25T17:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 client.15426 shutdown 2024-04-25T17:11:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 msgr2=0x5556968cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968cf130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0460005f70 tx=0x7f04600fd5c0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 shutdown_connections 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5556964b21c0 0x555696944100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55569651e1b0 0x5556968cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5556964fd4d0 0x5556968cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0443fff640 0 client.15426 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 --2- 172.21.15.96:0/1591936690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556964f52f0 0x5556968cf130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 >> 172.21.15.96:0/1591936690 conn(0x555696512300 msgr2=0x5556968bf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 shutdown_connections 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 1 -- 172.21.15.96:0/1591936690 wait complete. 2024-04-25T17:11:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.673+0000 7f0471970740 20 client.15426 trim_cache size 0 max 16384 2024-04-25T17:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:11:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f044c0afdf0 msgr2=0x7f044c0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f044c0afdf0 0x7f044c0b2300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f045c007fd0 tx=0x7f045c0072a0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 msgr2=0x7f046419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f046419a750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f046002aa00 tx=0x7f0460003cd0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 shutdown_connections 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f046410e0e0 0x7f046419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f044c0afdf0 0x7f044c0b2300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f046410b6f0 0x7f046419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 --2- 172.21.15.96:0/3979570897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0464104c90 0x7f046419a750 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 >> 172.21.15.96:0/3979570897 conn(0x7f04640fe260 msgr2=0x7f046410e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 shutdown_connections 2024-04-25T17:11:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:09.674+0000 7f046aabf640 1 -- 172.21.15.96:0/3979570897 wait complete. 2024-04-25T17:11:10.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:10 smithi096 ceph-mon[29131]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-04-25T17:11:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:10 smithi097 ceph-mon[32602]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-04-25T17:11:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:10 smithi171 ceph-mon[33189]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-04-25T17:11:12.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:12 smithi096 ceph-mon[29131]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T17:11:12.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:12 smithi097 ceph-mon[32602]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T17:11:12.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:12 smithi171 ceph-mon[33189]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T17:11:14.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:14 smithi096 ceph-mon[29131]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T17:11:14.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:14 smithi097 ceph-mon[32602]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T17:11:14.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:14 smithi171 ceph-mon[33189]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T17:11:16.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:16 smithi096 ceph-mon[29131]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T17:11:16.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:16 smithi097 ceph-mon[32602]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T17:11:16.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:16 smithi171 ceph-mon[33189]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T17:11:18.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:18 smithi096 ceph-mon[29131]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 597 B/s wr, 3 op/s 2024-04-25T17:11:18.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:18 smithi097 ceph-mon[32602]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 597 B/s wr, 3 op/s 2024-04-25T17:11:18.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:18 smithi171 ceph-mon[33189]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 597 B/s wr, 3 op/s 2024-04-25T17:11:20.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:20 smithi096 ceph-mon[29131]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:20.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:11:20.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:20.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:20 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:20 smithi097 ceph-mon[32602]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:11:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:20 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:20.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:20 smithi171 ceph-mon[33189]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:20.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:11:20.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:20.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:20 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:21 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:11:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:21 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:11:21.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:21 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:21.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:21 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:11:21.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:21 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:11:21.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:21 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:21.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:21 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:11:21.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:21 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:11:21.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:21 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:11:22.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:22 smithi096 ceph-mon[29131]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:22.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:22 smithi097 ceph-mon[32602]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 -- 172.21.15.96:0/2887518416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1031c0 msgr2=0x7f0ccc10f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 --2- 172.21.15.96:0/2887518416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1031c0 0x7f0ccc10f4f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009ca0 tx=0x7f0cc402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 -- 172.21.15.96:0/2887518416 shutdown_connections 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 --2- 172.21.15.96:0/2887518416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1031c0 0x7f0ccc10f4f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 --2- 172.21.15.96:0/2887518416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ccc108f00 0x7f0ccc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 --2- 172.21.15.96:0/2887518416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ccc108a60 0x7f0ccc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 -- 172.21.15.96:0/2887518416 >> 172.21.15.96:0/2887518416 conn(0x7f0ccc0781b0 msgr2=0x7f0ccc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 -- 172.21.15.96:0/2887518416 shutdown_connections 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 -- 172.21.15.96:0/2887518416 wait complete. 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 Processor -- start 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.776+0000 7f0cd1442640 1 -- start start 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0cd1442640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ccc1031c0 0x7f0ccc19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0cd1442640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ccc108a60 0x7f0ccc19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0cd1442640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc108f00 0x7f0ccc19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0cd1442640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc112ca0 con 0x7f0ccc108f00 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0cd1442640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc112b20 con 0x7f0ccc1031c0 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0cd1442640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc112e20 con 0x7f0ccc108a60 2024-04-25T17:11:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc108f00 0x7f0ccc19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc108f00 0x7f0ccc19c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43982/0 (socket says 172.21.15.96:43982) 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 -- 172.21.15.96:0/3736003398 learned_addr learned my addr 172.21.15.96:0/3736003398 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ccc1031c0 0x7f0ccc19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ccc108a60 0x7f0ccc19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 -- 172.21.15.96:0/3736003398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ccc108a60 msgr2=0x7f0ccc19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ccc108a60 0x7f0ccc19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 -- 172.21.15.96:0/3736003398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ccc1031c0 msgr2=0x7f0ccc19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ccc1031c0 0x7f0ccc19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 -- 172.21.15.96:0/3736003398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc1a6840 con 0x7f0ccc108f00 2024-04-25T17:11:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ccc1031c0 0x7f0ccc19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:11:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.777+0000 7f0ccb7fe640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc108f00 0x7f0ccc19c040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005cc0 tx=0x7f0cc402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.778+0000 7f0cc8ff9640 1 -- 172.21.15.96:0/3736003398 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4003bf0 con 0x7f0ccc108f00 2024-04-25T17:11:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.778+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc1a6ad0 con 0x7f0ccc108f00 2024-04-25T17:11:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.778+0000 7f0cc8ff9640 1 -- 172.21.15.96:0/3736003398 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc4003d90 con 0x7f0ccc108f00 2024-04-25T17:11:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.778+0000 7f0cc8ff9640 1 -- 172.21.15.96:0/3736003398 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc402fbc0 con 0x7f0ccc108f00 2024-04-25T17:11:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.778+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a7020 con 0x7f0ccc108f00 2024-04-25T17:11:22.783 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:22 smithi171 ceph-mon[33189]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.780+0000 7f0cd83d0740 1 Processor -- start 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.780+0000 7f0cc8ff9640 1 -- 172.21.15.96:0/3736003398 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f0cc402fd60 con 0x7f0ccc108f00 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cd83d0740 1 -- start start 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef45af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef46030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef46570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d2eeac460 con 0x564d2eba61b0 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d2ee66590 con 0x564d2eb7d2f0 2024-04-25T17:11:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564d2eea8420 con 0x564d2eb854d0 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef45af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef45af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43990/0 (socket says 172.21.15.96:43990) 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cc3fff640 1 -- 172.21.15.96:0/3673686279 learned_addr learned my addr 172.21.15.96:0/3673686279 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef46030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cc8ff9640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f0cb80afd70 0x7f0cb80b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef46570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.781+0000 7f0cc3fff640 1 -- 172.21.15.96:0/3673686279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 msgr2=0x564d2ef46570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef46570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc3fff640 1 -- 172.21.15.96:0/3673686279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 msgr2=0x564d2ef46030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef46030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0ccb7fe640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f0cb80afd70 0x7f0cb80b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc3fff640 1 -- 172.21.15.96:0/3673686279 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2ee730d0 con 0x564d2eba61b0 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc8ff9640 1 -- 172.21.15.96:0/3736003398 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f0cc41009b0 con 0x7f0ccc108f00 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef45af0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4010370 tx=0x7f0cb400c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bf3086211aea64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc17fa640 1 -- 172.21.15.96:0/3673686279 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4004030 con 0x564d2eba61b0 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc17fa640 1 -- 172.21.15.96:0/3673686279 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4005020 con 0x564d2eba61b0 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.782+0000 7f0cc17fa640 1 -- 172.21.15.96:0/3673686279 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4004b20 con 0x564d2eba61b0 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 -- 172.21.15.96:0/3673686279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 msgr2=0x564d2ef45af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef45af0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4010370 tx=0x7f0cb400c540 comp rx=0 tx=0).stop 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0ccb7fe640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f0cb80afd70 0x7f0cb80b2280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40fe100 tx=0x7f0cc40f7170 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 -- 172.21.15.96:0/3673686279 shutdown_connections 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef46570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef46030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/3673686279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef45af0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 -- 172.21.15.96:0/3673686279 >> 172.21.15.96:0/3673686279 conn(0x564d2eb9a300 msgr2=0x564d2ef422a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 -- 172.21.15.96:0/3673686279 shutdown_connections 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 -- 172.21.15.96:0/3673686279 wait complete. 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.783+0000 7f0cd83d0740 1 -- start start 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 1 Processor -- start 2024-04-25T17:11:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d2ef52c20 con 0x564d2eba61b0 2024-04-25T17:11:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d2ee33b60 con 0x564d2eb7d2f0 2024-04-25T17:11:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564d2ef52da0 con 0x564d2eb854d0 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cd83d0740 10 client.0 _ll_register_callbacks cb 0x564d2ee582b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44002/0 (socket says 172.21.15.96:44002) 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 -- 172.21.15.96:0/1545508849 learned_addr learned my addr 172.21.15.96:0/1545508849 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 -- 172.21.15.96:0/1545508849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 msgr2=0x564d2ef50720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef50720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 -- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 msgr2=0x564d2ef501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.784+0000 7f0cc3fff640 1 -- 172.21.15.96:0/1545508849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2ebbe210 con 0x564d2eba61b0 2024-04-25T17:11:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef561b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4010f70 tx=0x7f0cb400c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8cd572a5f3637c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cb27fc640 1 -- 172.21.15.96:0/1545508849 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4012660 con 0x564d2eba61b0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cb27fc640 1 -- 172.21.15.96:0/1545508849 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f0cb4012800 con 0x564d2eba61b0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb4005020 con 0x564d2eba61b0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cb27fc640 1 -- 172.21.15.96:0/1545508849 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0cb4012a00 con 0x564d2eba61b0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 10 client.15438 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564d2ec8a620 con 0x564d2eba61b0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cb27fc640 1 -- 172.21.15.96:0/1545508849 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4005020 con 0x564d2eba61b0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cb27fc640 1 -- 172.21.15.96:0/1545508849 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f0cb4033ae0 con 0x564d2eba61b0 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 10 client.15438 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 20 client.15438 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb64d80 0x564d2ef62690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 4 client.15438 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.785+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d2eeba070 con 0x564d2eb64d80 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.786+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb64d80 0x564d2ef62690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.786+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb64d80 0x564d2ef62690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3142d85f7236f34 server_cookie=272f7d08a0b97ef in_seq=0 out_seq=0 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.786+0000 7f0cb27fc640 10 client.15438 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cb27fc640 1 -- 172.21.15.96:0/1545508849 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d2eeba070 con 0x564d2eb64d80 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cb27fc640 10 client.15438 handle_command_reply: tid=0 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 client.15438 shutdown 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 msgr2=0x564d2ef561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef561b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4010f70 tx=0x7f0cb400c780 comp rx=0 tx=0).stop 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 shutdown_connections 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb64d80 0x564d2ef62690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef561b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef50720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cb27fc640 0 client.15438 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/1545508849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 >> 172.21.15.96:0/1545508849 conn(0x564d2eb9a300 msgr2=0x564d2ef458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 shutdown_connections 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 1 -- 172.21.15.96:0/1545508849 wait complete. 2024-04-25T17:11:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.787+0000 7f0cd83d0740 20 client.15438 trim_cache size 0 max 16384 2024-04-25T17:11:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.805+0000 7f0cd83d0740 1 Processor -- start 2024-04-25T17:11:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.805+0000 7f0cd83d0740 1 -- start start 2024-04-25T17:11:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef4f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef5c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef5c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ada00 con 0x564d2eba61b0 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1b0280 con 0x564d2eb7d2f0 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ae070 con 0x564d2eb854d0 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef5c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef5c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44006/0 (socket says 172.21.15.96:44006) 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 -- 172.21.15.96:0/484140413 learned_addr learned my addr 172.21.15.96:0/484140413 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef4f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef5c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 -- 172.21.15.96:0/484140413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 msgr2=0x564d2ef5c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef5c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 -- 172.21.15.96:0/484140413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 msgr2=0x564d2ef4f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef4f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 -- 172.21.15.96:0/484140413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2ef5cc20 con 0x564d2eba61b0 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.806+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef5c6e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x564d2ee32350 tx=0x7f0cb4005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8358bacddf3f7a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.807+0000 7f0cc0ff9640 1 -- 172.21.15.96:0/484140413 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb402c5c0 con 0x564d2eba61b0 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.807+0000 7f0cc0ff9640 1 -- 172.21.15.96:0/484140413 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4012660 con 0x564d2eba61b0 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.807+0000 7f0cc0ff9640 1 -- 172.21.15.96:0/484140413 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4004f40 con 0x564d2eba61b0 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.807+0000 7f0cd83d0740 1 -- 172.21.15.96:0/484140413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 msgr2=0x564d2ef5c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.807+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef5c6e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x564d2ee32350 tx=0x7f0cb4005540 comp rx=0 tx=0).stop 2024-04-25T17:11:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- 172.21.15.96:0/484140413 shutdown_connections 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2ef5c6e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2ef5c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/484140413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2ef4f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- 172.21.15.96:0/484140413 >> 172.21.15.96:0/484140413 conn(0x564d2eb9a300 msgr2=0x564d2ef52120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- 172.21.15.96:0/484140413 shutdown_connections 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- 172.21.15.96:0/484140413 wait complete. 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- start start 2024-04-25T17:11:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 Processor -- start 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2efbe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2efbeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2efc97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ae740 con 0x564d2eba61b0 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1afbb0 con 0x564d2eb7d2f0 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aee10 con 0x564d2eb854d0 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cd83d0740 10 client.0 _ll_register_callbacks cb 0x7f0cbc00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2efc97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2efc97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44018/0 (socket says 172.21.15.96:44018) 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.808+0000 7f0cc3fff640 1 -- 172.21.15.96:0/4240152446 learned_addr learned my addr 172.21.15.96:0/4240152446 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2efbe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2efbeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cc3fff640 1 -- 172.21.15.96:0/4240152446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 msgr2=0x564d2efbeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2efbeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cc3fff640 1 -- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 msgr2=0x564d2efbe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2efbe960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cc3fff640 1 -- 172.21.15.96:0/4240152446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2efc9d10 con 0x564d2eba61b0 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cc3fff640 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2efc97d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x564d2ef5b850 tx=0x7f0cb4011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6105ec8a13f5b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cb27fc640 1 -- 172.21.15.96:0/4240152446 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4012810 con 0x564d2eba61b0 2024-04-25T17:11:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cb27fc640 1 -- 172.21.15.96:0/4240152446 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f0cb4012db0 con 0x564d2eba61b0 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb4012660 con 0x564d2eba61b0 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cb27fc640 1 -- 172.21.15.96:0/4240152446 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0cb402c0b0 con 0x564d2eba61b0 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cd83d0740 10 client.15444 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.809+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x564d2efcade0 con 0x564d2eba61b0 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cb27fc640 1 -- 172.21.15.96:0/4240152446 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4012660 con 0x564d2eba61b0 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cb27fc640 1 -- 172.21.15.96:0/4240152446 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f0cb40041c0 con 0x564d2eba61b0 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cd83d0740 10 client.15444 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cd83d0740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cd83d0740 20 client.15444 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb3a1c0 0x564d2efcd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cd83d0740 4 client.15444 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:11:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d2eeb3ee0 con 0x564d2eb3a1c0 2024-04-25T17:11:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.810+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb3a1c0 0x564d2efcd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.811+0000 7f0ccaffd640 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb3a1c0 0x564d2efcd4b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e37c3d6ed55d08eb server_cookie=5a3539afcfe92001 in_seq=0 out_seq=0 2024-04-25T17:11:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.811+0000 7f0cb27fc640 10 client.15444 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cb27fc640 1 -- 172.21.15.96:0/4240152446 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d2eeb3ee0 con 0x564d2eb3a1c0 2024-04-25T17:11:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cb27fc640 10 client.15444 handle_command_reply: tid=0 2024-04-25T17:11:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 client.15444 shutdown 2024-04-25T17:11:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 msgr2=0x564d2efc97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2efc97d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x564d2ef5b850 tx=0x7f0cb4011f40 comp rx=0 tx=0).stop 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 shutdown_connections 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x564d2eb3a1c0 0x564d2efcd4b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2eba61b0 0x564d2efc97d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cb27fc640 0 client.15444 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564d2eb854d0 0x564d2efbeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 --2- 172.21.15.96:0/4240152446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d2eb7d2f0 0x564d2efbe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 >> 172.21.15.96:0/4240152446 conn(0x564d2eb9a300 msgr2=0x564d2ef51e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 shutdown_connections 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 1 -- 172.21.15.96:0/4240152446 wait complete. 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.815+0000 7f0cd83d0740 20 client.15444 trim_cache size 0 max 16384 2024-04-25T17:11:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:11:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:11:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:11:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f0cb80afd70 msgr2=0x7f0cb80b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f0cb80afd70 0x7f0cb80b2280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40fe100 tx=0x7f0cc40f7170 comp rx=0 tx=0).stop 2024-04-25T17:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc108f00 msgr2=0x7f0ccc19c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc108f00 0x7f0ccc19c040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005cc0 tx=0x7f0cc402f380 comp rx=0 tx=0).stop 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 shutdown_connections 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc108f00 0x7f0ccc19c040 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ccc108a60 0x7f0ccc19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f0cb80afd70 0x7f0cb80b2280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 --2- 172.21.15.96:0/3736003398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ccc1031c0 0x7f0ccc19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 >> 172.21.15.96:0/3736003398 conn(0x7f0ccc0781b0 msgr2=0x7f0ccc100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 shutdown_connections 2024-04-25T17:11:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:22.816+0000 7f0cd1442640 1 -- 172.21.15.96:0/3736003398 wait complete. 2024-04-25T17:11:24.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:24 smithi096 ceph-mon[29131]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:24.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:24 smithi097 ceph-mon[32602]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:24.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:24 smithi171 ceph-mon[33189]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:26.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:26 smithi096 ceph-mon[29131]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:26.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:26 smithi097 ceph-mon[32602]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:26.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:26 smithi171 ceph-mon[33189]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:28.690 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:28 smithi096 ceph-mon[29131]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:28.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:28 smithi097 ceph-mon[32602]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:28.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:28 smithi171 ceph-mon[33189]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:30.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:30 smithi096 ceph-mon[29131]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:30.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:30 smithi097 ceph-mon[32602]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:30.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:30 smithi171 ceph-mon[33189]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:32.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:32 smithi096 ceph-mon[29131]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:32.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:32 smithi097 ceph-mon[32602]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:32.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:32 smithi171 ceph-mon[33189]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:34.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:34 smithi096 ceph-mon[29131]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:34.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:34 smithi097 ceph-mon[32602]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:34.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:34 smithi171 ceph-mon[33189]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 -- 172.21.15.96:0/648096489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c103820 msgr2=0x7f8d5c10f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 --2- 172.21.15.96:0/648096489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c103820 0x7f8d5c10f590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009ca0 tx=0x7f8d5402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 -- 172.21.15.96:0/648096489 shutdown_connections 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 --2- 172.21.15.96:0/648096489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c103820 0x7f8d5c10f590 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 --2- 172.21.15.96:0/648096489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5c108f10 0x7f8d5c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 --2- 172.21.15.96:0/648096489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d5c108a70 0x7f8d5c102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 -- 172.21.15.96:0/648096489 >> 172.21.15.96:0/648096489 conn(0x7f8d5c0fd8b0 msgr2=0x7f8d5c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 -- 172.21.15.96:0/648096489 shutdown_connections 2024-04-25T17:11:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.963+0000 7f8d6400d640 1 -- 172.21.15.96:0/648096489 wait complete. 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.964+0000 7f8d6400d640 1 Processor -- start 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.964+0000 7f8d6400d640 1 -- start start 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.964+0000 7f8d6400d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d5c103820 0x7f8d5c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.964+0000 7f8d6400d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c108a70 0x7f8d5c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.964+0000 7f8d6400d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5c108f10 0x7f8d5c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.964+0000 7f8d615a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c108a70 0x7f8d5c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.964+0000 7f8d615a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c108a70 0x7f8d5c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43316/0 (socket says 172.21.15.96:43316) 2024-04-25T17:11:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d615a8640 1 -- 172.21.15.96:0/2303511795 learned_addr learned my addr 172.21.15.96:0/2303511795 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c112cd0 con 0x7f8d5c108a70 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c112b50 con 0x7f8d5c108f10 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c112e50 con 0x7f8d5c103820 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d61da9640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d5c103820 0x7f8d5c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d625aa640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5c108f10 0x7f8d5c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d61da9640 1 -- 172.21.15.96:0/2303511795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5c108f10 msgr2=0x7f8d5c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d61da9640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5c108f10 0x7f8d5c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d61da9640 1 -- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c108a70 msgr2=0x7f8d5c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d61da9640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c108a70 0x7f8d5c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d61da9640 1 -- 172.21.15.96:0/2303511795 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c19f360 con 0x7f8d5c103820 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d615a8640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c108a70 0x7f8d5c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d625aa640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5c108f10 0x7f8d5c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:11:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.965+0000 7f8d61da9640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d5c103820 0x7f8d5c19a500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009b20 tx=0x7f8d5800b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.966+0000 7f8d52ffd640 1 -- 172.21.15.96:0/2303511795 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5801e070 con 0x7f8d5c103820 2024-04-25T17:11:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.966+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c19f650 con 0x7f8d5c103820 2024-04-25T17:11:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.966+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c104e20 con 0x7f8d5c103820 2024-04-25T17:11:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.966+0000 7f8d52ffd640 1 -- 172.21.15.96:0/2303511795 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5800bd00 con 0x7f8d5c103820 2024-04-25T17:11:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.966+0000 7f8d52ffd640 1 -- 172.21.15.96:0/2303511795 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58019450 con 0x7f8d5c103820 2024-04-25T17:11:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.969+0000 7f8d6af95740 1 Processor -- start 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.969+0000 7f8d6af95740 1 -- start start 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.969+0000 7f8d52ffd640 1 -- 172.21.15.96:0/2303511795 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 147753+0+0 (secure 0 0 0) 0x7f8d58021070 con 0x7f8d5c103820 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98f041b0 0x562b992a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ee34d0 0x562b992a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d6af95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562b9920a460 con 0x562b98ee34d0 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d6af95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562b991c4590 con 0x562b98edb2f0 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d6af95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562b99206420 con 0x562b98f041b0 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d615a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ee34d0 0x562b992a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d615a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ee34d0 0x562b992a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43324/0 (socket says 172.21.15.96:43324) 2024-04-25T17:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d615a8640 1 -- 172.21.15.96:0/3000036874 learned_addr learned my addr 172.21.15.96:0/3000036874 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d61da9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.970+0000 7f8d625aa640 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98f041b0 0x562b992a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d52ffd640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8d480afdf0 0x7f8d480b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d615a8640 1 -- 172.21.15.96:0/3000036874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98f041b0 msgr2=0x562b992a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98f041b0 0x562b992a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d615a8640 1 -- 172.21.15.96:0/3000036874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 msgr2=0x562b992a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d615a8640 1 -- 172.21.15.96:0/3000036874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b991d10d0 con 0x562b98ee34d0 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d625aa640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8d480afdf0 0x7f8d480b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.971+0000 7f8d52ffd640 1 -- 172.21.15.96:0/2303511795 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f8d580e2bc0 con 0x7f8d5c103820 2024-04-25T17:11:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.972+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ee34d0 0x562b992a4570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0058c0 tx=0x7f8d4c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba24b7809886da6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.972+0000 7f8d625aa640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8d480afdf0 0x7f8d480b2300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d5402ade0 tx=0x7f8d54005fd0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.972+0000 7f8d47fff640 1 -- 172.21.15.96:0/3000036874 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c005020 con 0x562b98ee34d0 2024-04-25T17:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.972+0000 7f8d47fff640 1 -- 172.21.15.96:0/3000036874 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d4c0047a0 con 0x562b98ee34d0 2024-04-25T17:11:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.972+0000 7f8d47fff640 1 -- 172.21.15.96:0/3000036874 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00f840 con 0x562b98ee34d0 2024-04-25T17:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 -- 172.21.15.96:0/3000036874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ee34d0 msgr2=0x562b992a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ee34d0 0x562b992a4570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0058c0 tx=0x7f8d4c0119a0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 -- 172.21.15.96:0/3000036874 shutdown_connections 2024-04-25T17:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ee34d0 0x562b992a4570 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3000036874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98f041b0 0x562b992a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 -- 172.21.15.96:0/3000036874 >> 172.21.15.96:0/3000036874 conn(0x562b98ef8300 msgr2=0x562b992a02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 -- 172.21.15.96:0/3000036874 shutdown_connections 2024-04-25T17:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.973+0000 7f8d6af95740 1 -- 172.21.15.96:0/3000036874 wait complete. 2024-04-25T17:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.974+0000 7f8d6af95740 1 -- start start 2024-04-25T17:11:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.974+0000 7f8d6af95740 1 Processor -- start 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ee34d0 0x562b992ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98f041b0 0x562b992ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d6af95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562b992b0c20 con 0x562b98f041b0 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d6af95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562b99191b60 con 0x562b98ee34d0 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d6af95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562b992b0da0 con 0x562b98edb2f0 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d6af95740 10 client.0 _ll_register_callbacks cb 0x562b991b62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98f041b0 0x562b992ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98f041b0 0x562b992ba0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43326/0 (socket says 172.21.15.96:43326) 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 -- 172.21.15.96:0/3240411811 learned_addr learned my addr 172.21.15.96:0/3240411811 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ee34d0 0x562b992ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d61da9640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 -- 172.21.15.96:0/3240411811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 msgr2=0x562b992ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 -- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ee34d0 msgr2=0x562b992ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ee34d0 0x562b992ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d625aa640 1 -- 172.21.15.96:0/3240411811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b98f1c210 con 0x562b98f041b0 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.975+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ee34d0 0x562b992ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d625aa640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98f041b0 0x562b992ba0d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8d540319b0 tx=0x7f8d5402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0e7b40b9456877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d457fa640 1 -- 172.21.15.96:0/3240411811 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d540047c0 con 0x562b98f041b0 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d457fa640 1 -- 172.21.15.96:0/3240411811 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f8d5402f610 con 0x562b98f041b0 2024-04-25T17:11:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d4c0047a0 con 0x562b98f041b0 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d457fa640 1 -- 172.21.15.96:0/3240411811 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8d5402f810 con 0x562b98f041b0 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d6af95740 10 client.15456 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.976+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x562b98fe8620 con 0x562b98f041b0 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d457fa640 1 -- 172.21.15.96:0/3240411811 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d4c0047a0 con 0x562b98f041b0 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d457fa640 1 -- 172.21.15.96:0/3240411811 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f8d54030860 con 0x562b98f041b0 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d6af95740 10 client.15456 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d6af95740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d6af95740 20 client.15456 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ec2d80 0x562b992c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d6af95740 4 client.15456 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b99218070 con 0x562b98ec2d80 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.977+0000 7f8d61da9640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ec2d80 0x562b992c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.978+0000 7f8d457fa640 10 client.15456 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.978+0000 7f8d61da9640 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ec2d80 0x562b992c0710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a998fe625068697b server_cookie=bc48dd0b362f9626 in_seq=0 out_seq=0 2024-04-25T17:11:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.978+0000 7f8d457fa640 1 -- 172.21.15.96:0/3240411811 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b99218070 con 0x562b98ec2d80 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.978+0000 7f8d457fa640 10 client.15456 handle_command_reply: tid=0 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.978+0000 7f8d6af95740 1 client.15456 shutdown 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98f041b0 msgr2=0x562b992ba0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98f041b0 0x562b992ba0d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8d540319b0 tx=0x7f8d5402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 shutdown_connections 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ec2d80 0x562b992c0710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98f041b0 0x562b992ba0d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ee34d0 0x562b992ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d457fa640 0 client.15456 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3240411811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 >> 172.21.15.96:0/3240411811 conn(0x562b98ef8300 msgr2=0x562b992a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 shutdown_connections 2024-04-25T17:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 1 -- 172.21.15.96:0/3240411811 wait complete. 2024-04-25T17:11:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:35.979+0000 7f8d6af95740 20 client.15456 trim_cache size 0 max 16384 2024-04-25T17:11:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 Processor -- start 2024-04-25T17:11:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 -- start start 2024-04-25T17:11:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ef8300 0x562b992a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98ec2d80 0x562b992b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4000c730 con 0x562b98ef8300 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4000efb0 con 0x562b98edb2f0 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d6af95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4000cda0 con 0x562b98ec2d80 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d61da9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ef8300 0x562b992a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d61da9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ef8300 0x562b992a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43340/0 (socket says 172.21.15.96:43340) 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d615a8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98ec2d80 0x562b992b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d61da9640 1 -- 172.21.15.96:0/2638033657 learned_addr learned my addr 172.21.15.96:0/2638033657 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.009+0000 7f8d625aa640 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d615a8640 1 -- 172.21.15.96:0/2638033657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 msgr2=0x562b992c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d615a8640 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d615a8640 1 -- 172.21.15.96:0/2638033657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ef8300 msgr2=0x562b992a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d615a8640 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ef8300 0x562b992a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d615a8640 1 -- 172.21.15.96:0/2638033657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b992a0e90 con 0x562b98ec2d80 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d61da9640 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ef8300 0x562b992a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d625aa640 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d615a8640 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98ec2d80 0x562b992b9cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c01b040 tx=0x7f8d4c009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54fec489f038cbac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d477fe640 1 -- 172.21.15.96:0/2638033657 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c004100 con 0x562b98ec2d80 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d477fe640 1 -- 172.21.15.96:0/2638033657 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d4c0042a0 con 0x562b98ec2d80 2024-04-25T17:11:36.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.010+0000 7f8d477fe640 1 -- 172.21.15.96:0/2638033657 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00f840 con 0x562b98ec2d80 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.011+0000 7f8d6af95740 1 -- 172.21.15.96:0/2638033657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98ec2d80 msgr2=0x562b992b9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.011+0000 7f8d6af95740 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98ec2d80 0x562b992b9cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c01b040 tx=0x7f8d4c009530 comp rx=0 tx=0).stop 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.011+0000 7f8d6af95740 1 -- 172.21.15.96:0/2638033657 shutdown_connections 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.011+0000 7f8d6af95740 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98edb2f0 0x562b992c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.011+0000 7f8d6af95740 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98ec2d80 0x562b992b9cb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.011+0000 7f8d6af95740 1 --2- 172.21.15.96:0/2638033657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ef8300 0x562b992a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.011+0000 7f8d6af95740 1 -- 172.21.15.96:0/2638033657 >> 172.21.15.96:0/2638033657 conn(0x562b992ae790 msgr2=0x562b9929fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.012+0000 7f8d6af95740 1 -- 172.21.15.96:0/2638033657 shutdown_connections 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.012+0000 7f8d6af95740 1 -- 172.21.15.96:0/2638033657 wait complete. 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.012+0000 7f8d6af95740 1 -- start start 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.012+0000 7f8d6af95740 1 Processor -- start 2024-04-25T17:11:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ec2d80 0x562b992b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d6af95740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ef8300 0x562b992b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d6af95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4000d470 con 0x562b98ec2d80 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d6af95740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4000e8e0 con 0x562b98ef8300 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d6af95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4000db40 con 0x562b98edb2f0 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ec2d80 0x562b992b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ec2d80 0x562b992b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43354/0 (socket says 172.21.15.96:43354) 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d6af95740 10 client.0 _ll_register_callbacks cb 0x7f8d58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 -- 172.21.15.96:0/3967725422 learned_addr learned my addr 172.21.15.96:0/3967725422 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d625aa640 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ef8300 0x562b992b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 -- 172.21.15.96:0/3967725422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 msgr2=0x562b992b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 -- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ef8300 msgr2=0x562b992b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ef8300 0x562b992b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.013+0000 7f8d61da9640 1 -- 172.21.15.96:0/3967725422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b99327cc0 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d61da9640 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ec2d80 0x562b992b1e60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8d580e0920 tx=0x7f8d580a63c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc650aa229e131f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d457fa640 1 -- 172.21.15.96:0/3967725422 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d580e0680 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d457fa640 1 -- 172.21.15.96:0/3967725422 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f8d580a9d80 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d4c0042a0 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d457fa640 1 -- 172.21.15.96:0/3967725422 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8d580ab220 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d6af95740 10 client.15462 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x562b99328f70 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d457fa640 1 -- 172.21.15.96:0/3967725422 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d4c0042a0 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.014+0000 7f8d457fa640 1 -- 172.21.15.96:0/3967725422 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f8d580a97f0 con 0x562b98ec2d80 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d6af95740 10 client.15462 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d6af95740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d6af95740 20 client.15462 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ee34d0 0x562b9932b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d6af95740 4 client.15462 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:11:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b99211ee0 con 0x562b98ee34d0 2024-04-25T17:11:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ee34d0 0x562b9932b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d457fa640 10 client.15462 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.015+0000 7f8d615a8640 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ee34d0 0x562b9932b680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784fb51590b61c5e server_cookie=b2a320581cb725dd in_seq=0 out_seq=0 2024-04-25T17:11:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.019+0000 7f8d457fa640 1 -- 172.21.15.96:0/3967725422 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b99211ee0 con 0x562b98ee34d0 2024-04-25T17:11:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.019+0000 7f8d457fa640 10 client.15462 handle_command_reply: tid=0 2024-04-25T17:11:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.019+0000 7f8d6af95740 1 client.15462 shutdown 2024-04-25T17:11:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.019+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ec2d80 msgr2=0x562b992b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.019+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ec2d80 0x562b992b1e60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8d580e0920 tx=0x7f8d580a63c0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 shutdown_connections 2024-04-25T17:11:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x562b98ee34d0 0x562b9932b680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562b98ef8300 0x562b992b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562b98edb2f0 0x562b992b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 --2- 172.21.15.96:0/3967725422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562b98ec2d80 0x562b992b1e60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 >> 172.21.15.96:0/3967725422 conn(0x562b992ae790 msgr2=0x562b992a3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d457fa640 0 client.15462 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 shutdown_connections 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 1 -- 172.21.15.96:0/3967725422 wait complete. 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6af95740 20 client.15462 trim_cache size 0 max 16384 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:11:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:11:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8d480afdf0 msgr2=0x7f8d480b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6400d640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8d480afdf0 0x7f8d480b2300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d5402ade0 tx=0x7f8d54005fd0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d5c103820 msgr2=0x7f8d5c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.020+0000 7f8d6400d640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d5c103820 0x7f8d5c19a500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009b20 tx=0x7f8d5800b2c0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 shutdown_connections 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8d480afdf0 0x7f8d480b2300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5c108f10 0x7f8d5c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d5c108a70 0x7f8d5c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 --2- 172.21.15.96:0/2303511795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d5c103820 0x7f8d5c19a500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 >> 172.21.15.96:0/2303511795 conn(0x7f8d5c0fd8b0 msgr2=0x7f8d5c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 shutdown_connections 2024-04-25T17:11:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:36.021+0000 7f8d6400d640 1 -- 172.21.15.96:0/2303511795 wait complete. 2024-04-25T17:11:36.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:36 smithi096 ceph-mon[29131]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:36.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:36 smithi097 ceph-mon[32602]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:36.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:36 smithi171 ceph-mon[33189]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:38.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:38 smithi096 ceph-mon[29131]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:38.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:38 smithi097 ceph-mon[32602]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:38.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:38 smithi171 ceph-mon[33189]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:40.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:40 smithi096 ceph-mon[29131]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:40.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:40 smithi097 ceph-mon[32602]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:40.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:40 smithi171 ceph-mon[33189]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:11:42.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:42 smithi096 ceph-mon[29131]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:42.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:42 smithi097 ceph-mon[32602]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:42.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:42 smithi171 ceph-mon[33189]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:44.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:44 smithi096 ceph-mon[29131]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:44.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:44 smithi097 ceph-mon[32602]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:44.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:44 smithi171 ceph-mon[33189]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:46.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:46 smithi096 ceph-mon[29131]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:46.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:46 smithi097 ceph-mon[32602]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:46.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:46 smithi171 ceph-mon[33189]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:11:48.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:48 smithi096 ceph-mon[29131]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:48.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:48 smithi096 ceph-mon[29131]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T17:11:48.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:48 smithi097 ceph-mon[32602]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:48.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:48 smithi097 ceph-mon[32602]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T17:11:48.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:48 smithi171 ceph-mon[33189]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:48.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:48 smithi171 ceph-mon[33189]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T17:11:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 -- 172.21.15.96:0/3531034229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7144108f00 msgr2=0x7f7144102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 --2- 172.21.15.96:0/3531034229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7144108f00 0x7f7144102b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7134009900 tx=0x7f713402a410 comp rx=0 tx=0).stop 2024-04-25T17:11:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 -- 172.21.15.96:0/3531034229 shutdown_connections 2024-04-25T17:11:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 --2- 172.21.15.96:0/3531034229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71441031c0 0x7f714410f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 --2- 172.21.15.96:0/3531034229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7144108f00 0x7f7144102b50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 --2- 172.21.15.96:0/3531034229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7144108a60 0x7f7144102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 -- 172.21.15.96:0/3531034229 >> 172.21.15.96:0/3531034229 conn(0x7f71440781b0 msgr2=0x7f71440785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 -- 172.21.15.96:0/3531034229 shutdown_connections 2024-04-25T17:11:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.165+0000 7f714c467640 1 -- 172.21.15.96:0/3531034229 wait complete. 2024-04-25T17:11:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.166+0000 7f714c467640 1 Processor -- start 2024-04-25T17:11:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.166+0000 7f714c467640 1 -- start start 2024-04-25T17:11:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714c467640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71441031c0 0x7f71441996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714c467640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7144108a60 0x7f7144199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714c467640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7144108f00 0x7f714419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714c467640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7144112cb0 con 0x7f71441031c0 2024-04-25T17:11:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714c467640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7144112b30 con 0x7f7144108f00 2024-04-25T17:11:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714c467640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7144112e30 con 0x7f7144108a60 2024-04-25T17:11:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71441031c0 0x7f71441996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71441031c0 0x7f71441996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37954/0 (socket says 172.21.15.96:37954) 2024-04-25T17:11:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 -- 172.21.15.96:0/2711873395 learned_addr learned my addr 172.21.15.96:0/2711873395 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714aa04640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7144108f00 0x7f714419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f7149a02640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7144108a60 0x7f7144199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 -- 172.21.15.96:0/2711873395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7144108a60 msgr2=0x7f7144199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7144108a60 0x7f7144199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 -- 172.21.15.96:0/2711873395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7144108f00 msgr2=0x7f714419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7144108f00 0x7f714419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.167+0000 7f714a203640 1 -- 172.21.15.96:0/2711873395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714419e510 con 0x7f71441031c0 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.168+0000 7f714a203640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71441031c0 0x7f71441996b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f714000e9f0 tx=0x7f714000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.168+0000 7f713b7fe640 1 -- 172.21.15.96:0/2711873395 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714000ccc0 con 0x7f71441031c0 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.168+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714419e800 con 0x7f71441031c0 2024-04-25T17:11:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.168+0000 7f713b7fe640 1 -- 172.21.15.96:0/2711873395 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71400108e0 con 0x7f71441031c0 2024-04-25T17:11:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.168+0000 7f713b7fe640 1 -- 172.21.15.96:0/2711873395 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140018860 con 0x7f71441031c0 2024-04-25T17:11:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.168+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714406a3c0 con 0x7f71441031c0 2024-04-25T17:11:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.171+0000 7f71533fb740 1 Processor -- start 2024-04-25T17:11:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.171+0000 7f71533fb740 1 -- start start 2024-04-25T17:11:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f71533fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf335af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f71533fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef6d2f0 0x55d6cf336030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f71533fb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef754d0 0x55d6cf336570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f71533fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d6cf29c460 con 0x55d6cef6d2f0 2024-04-25T17:11:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f71533fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6cf256590 con 0x55d6cef961b0 2024-04-25T17:11:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f71533fb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d6cf298420 con 0x55d6cef754d0 2024-04-25T17:11:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f714aa04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf335af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f7149a02640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef754d0 0x55d6cf336570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f714aa04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf335af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:56442/0 (socket says 172.21.15.96:56442) 2024-04-25T17:11:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.172+0000 7f714aa04640 1 -- 172.21.15.96:0/2826220350 learned_addr learned my addr 172.21.15.96:0/2826220350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f713b7fe640 1 -- 172.21.15.96:0/2711873395 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7140018a00 con 0x7f71441031c0 2024-04-25T17:11:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f7149a02640 1 -- 172.21.15.96:0/2826220350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 msgr2=0x55d6cf335af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f7149a02640 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf335af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f7149a02640 1 -- 172.21.15.96:0/2826220350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef6d2f0 msgr2=0x55d6cf336030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f714a203640 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef6d2f0 0x55d6cf336030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f7149a02640 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef6d2f0 0x55d6cf336030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f7149a02640 1 -- 172.21.15.96:0/2826220350 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6cf2630d0 con 0x55d6cef754d0 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f714a203640 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef6d2f0 0x55d6cf336030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f713b7fe640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f71300afc50 0x7f71300b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f7149a02640 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef754d0 0x55d6cf336570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7134009230 tx=0x7f71340040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e875d41d7fd1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f714aa04640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f71300afc50 0x7f71300b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.173+0000 7f713b7fe640 1 -- 172.21.15.96:0/2711873395 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f71400df2a0 con 0x7f71441031c0 2024-04-25T17:11:49.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.174+0000 7f7138bfe640 1 -- 172.21.15.96:0/2826220350 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134035040 con 0x55d6cef754d0 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.174+0000 7f7138bfe640 1 -- 172.21.15.96:0/2826220350 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71340054f0 con 0x55d6cef754d0 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.174+0000 7f7138bfe640 1 -- 172.21.15.96:0/2826220350 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71340035f0 con 0x55d6cef754d0 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 -- 172.21.15.96:0/2826220350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef754d0 msgr2=0x55d6cf336570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f714aa04640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f71300afc50 0x7f71300b2160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55d6cf332c60 tx=0x7f713c011040 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef754d0 0x55d6cf336570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7134009230 tx=0x7f71340040c0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 -- 172.21.15.96:0/2826220350 shutdown_connections 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef754d0 0x55d6cf336570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef6d2f0 0x55d6cf336030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 --2- 172.21.15.96:0/2826220350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf335af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 -- 172.21.15.96:0/2826220350 >> 172.21.15.96:0/2826220350 conn(0x55d6cef8a300 msgr2=0x55d6cf3322a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 -- 172.21.15.96:0/2826220350 shutdown_connections 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 -- 172.21.15.96:0/2826220350 wait complete. 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.175+0000 7f71533fb740 1 -- start start 2024-04-25T17:11:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 1 Processor -- start 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf341b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef754d0 0x55d6cf3420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef961b0 0x55d6cf340010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d6cf342bc0 con 0x55d6cef961b0 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d6cf223b60 con 0x55d6cef754d0 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d6cf342d40 con 0x55d6cef6d2f0 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f71533fb740 10 client.0 _ll_register_callbacks cb 0x55d6cf2482b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f714aa04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef961b0 0x55d6cf340010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f7149a02640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef754d0 0x55d6cf3420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f7149a02640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef754d0 0x55d6cf3420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:56450/0 (socket says 172.21.15.96:56450) 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f714aa04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef961b0 0x55d6cf340010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37980/0 (socket says 172.21.15.96:37980) 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f7149a02640 1 -- 172.21.15.96:0/4056797637 learned_addr learned my addr 172.21.15.96:0/4056797637 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.176+0000 7f714a203640 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf341b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f7149a02640 1 -- 172.21.15.96:0/4056797637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 msgr2=0x55d6cf341b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f7149a02640 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf341b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f7149a02640 1 -- 172.21.15.96:0/4056797637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef961b0 msgr2=0x55d6cf340010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f7149a02640 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef961b0 0x55d6cf340010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f7149a02640 1 -- 172.21.15.96:0/4056797637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6cefae210 con 0x55d6cef754d0 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f7149a02640 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef754d0 0x55d6cf3420b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7134009900 tx=0x7f7134004ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d649309566bdb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f712dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f712dffb640 1 -- 172.21.15.96:0/4056797637 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134035040 con 0x55d6cef754d0 2024-04-25T17:11:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71340054f0 con 0x55d6cef754d0 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f712dffb640 1 -- 172.21.15.96:0/4056797637 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f7134005000 con 0x55d6cef754d0 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f712dffb640 1 -- 172.21.15.96:0/4056797637 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f713405a050 con 0x55d6cef754d0 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f71533fb740 10 client.25085 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.177+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d6cf07a620 con 0x55d6cef754d0 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f712dffb640 1 -- 172.21.15.96:0/4056797637 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71340054f0 con 0x55d6cef754d0 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f712dffb640 1 -- 172.21.15.96:0/4056797637 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f7134055030 con 0x55d6cef754d0 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f71533fb740 10 client.25085 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f71533fb740 10 client.25085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f71533fb740 20 client.25085 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f71533fb740 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cef54d80 0x55d6cf352330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f71533fb740 4 client.25085 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6cf2aa070 con 0x55d6cef54d80 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.178+0000 7f714a203640 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cef54d80 0x55d6cf352330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.179+0000 7f712dffb640 10 client.25085 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.179+0000 7f714a203640 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cef54d80 0x55d6cf352330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e50771270314398 server_cookie=2ec2ecb34378b801 in_seq=0 out_seq=0 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.179+0000 7f712dffb640 1 -- 172.21.15.96:0/4056797637 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6cf2aa070 con 0x55d6cef54d80 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.179+0000 7f712dffb640 10 client.25085 handle_command_reply: tid=0 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.179+0000 7f71533fb740 1 client.25085 shutdown 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef754d0 msgr2=0x55d6cf3420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef754d0 0x55d6cf3420b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7134009900 tx=0x7f7134004ea0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 shutdown_connections 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cef54d80 0x55d6cf352330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef961b0 0x55d6cf340010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f712dffb640 0 client.25085 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef754d0 0x55d6cf3420b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 --2- 172.21.15.96:0/4056797637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf341b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 >> 172.21.15.96:0/4056797637 conn(0x55d6cef8a300 msgr2=0x55d6cf337810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 shutdown_connections 2024-04-25T17:11:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 1 -- 172.21.15.96:0/4056797637 wait complete. 2024-04-25T17:11:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.180+0000 7f71533fb740 20 client.25085 trim_cache size 0 max 16384 2024-04-25T17:11:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.195+0000 7f71533fb740 1 Processor -- start 2024-04-25T17:11:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.195+0000 7f71533fb740 1 -- start start 2024-04-25T17:11:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f71533fb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf33eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f71533fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef754d0 0x55d6cf355350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f71533fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf355890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef754d0 0x55d6cf355350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef754d0 0x55d6cf355350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37988/0 (socket says 172.21.15.96:37988) 2024-04-25T17:11:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 -- 172.21.15.96:0/1209061557 learned_addr learned my addr 172.21.15.96:0/1209061557 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71441afbd0 con 0x55d6cef754d0 2024-04-25T17:11:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b2450 con 0x55d6cef961b0 2024-04-25T17:11:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b0240 con 0x55d6cef6d2f0 2024-04-25T17:11:49.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f714a203640 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf33eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f714aa04640 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf355890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 -- 172.21.15.96:0/1209061557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 msgr2=0x55d6cf33eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf33eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 -- 172.21.15.96:0/1209061557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 msgr2=0x55d6cf355890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf355890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f7149a02640 1 -- 172.21.15.96:0/1209061557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6cf355dd0 con 0x55d6cef754d0 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.196+0000 7f714aa04640 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf355890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.197+0000 7f7149a02640 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef754d0 0x55d6cf355350 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7134004010 tx=0x7f7134005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc36defafbfaba66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.197+0000 7f712ffff640 1 -- 172.21.15.96:0/1209061557 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134035040 con 0x55d6cef754d0 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.197+0000 7f712ffff640 1 -- 172.21.15.96:0/1209061557 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7134005850 con 0x55d6cef754d0 2024-04-25T17:11:49.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.197+0000 7f712ffff640 1 -- 172.21.15.96:0/1209061557 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134005450 con 0x55d6cef754d0 2024-04-25T17:11:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef754d0 msgr2=0x55d6cf355350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef754d0 0x55d6cf355350 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7134004010 tx=0x7f7134005ec0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 shutdown_connections 2024-04-25T17:11:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef961b0 0x55d6cf355890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef754d0 0x55d6cf355350 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 --2- 172.21.15.96:0/1209061557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef6d2f0 0x55d6cf33eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 >> 172.21.15.96:0/1209061557 conn(0x55d6cef8a300 msgr2=0x55d6cf33e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 shutdown_connections 2024-04-25T17:11:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 -- 172.21.15.96:0/1209061557 wait complete. 2024-04-25T17:11:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 -- start start 2024-04-25T17:11:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.198+0000 7f71533fb740 1 Processor -- start 2024-04-25T17:11:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f71533fb740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef6d2f0 0x55d6cf355890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f71533fb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef961b0 0x55d6cf352c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f71533fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef2a1c0 0x55d6cf3532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f71533fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b0910 con 0x55d6cef2a1c0 2024-04-25T17:11:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f71533fb740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b1d80 con 0x55d6cef6d2f0 2024-04-25T17:11:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f71533fb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71441b0fe0 con 0x55d6cef961b0 2024-04-25T17:11:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f71533fb740 10 client.0 _ll_register_callbacks cb 0x7f713c009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:11:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714aa04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef2a1c0 0x55d6cf3532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714aa04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef2a1c0 0x55d6cf3532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38000/0 (socket says 172.21.15.96:38000) 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714aa04640 1 -- 172.21.15.96:0/3907137959 learned_addr learned my addr 172.21.15.96:0/3907137959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714a203640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef6d2f0 0x55d6cf355890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f7149a02640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef961b0 0x55d6cf352c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714a203640 1 -- 172.21.15.96:0/3907137959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef961b0 msgr2=0x55d6cf352c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714a203640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef961b0 0x55d6cf352c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714a203640 1 -- 172.21.15.96:0/3907137959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef2a1c0 msgr2=0x55d6cf3532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:11:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714a203640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef2a1c0 0x55d6cf3532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714a203640 1 -- 172.21.15.96:0/3907137959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6cf353a10 con 0x55d6cef6d2f0 2024-04-25T17:11:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.199+0000 7f714aa04640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef2a1c0 0x55d6cf3532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:11:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f714a203640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef6d2f0 0x55d6cf355890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71400dca00 tx=0x7f7140009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28c4ee7e8f24b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:11:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f712dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T17:11:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f712dffb640 1 -- 172.21.15.96:0/3907137959 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71400b02b0 con 0x55d6cef6d2f0 2024-04-25T17:11:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f712dffb640 1 -- 172.21.15.96:0/3907137959 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f71400a3790 con 0x55d6cef6d2f0 2024-04-25T17:11:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7134005850 con 0x55d6cef6d2f0 2024-04-25T17:11:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f712dffb640 1 -- 172.21.15.96:0/3907137959 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f71400aeb80 con 0x55d6cef6d2f0 2024-04-25T17:11:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f71533fb740 10 client.25091 fetch_fsmap learned FSMap version 18 2024-04-25T17:11:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d6cf3ba4f0 con 0x55d6cef6d2f0 2024-04-25T17:11:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f712dffb640 1 -- 172.21.15.96:0/3907137959 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71400aed20 con 0x55d6cef6d2f0 2024-04-25T17:11:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f712dffb640 1 -- 172.21.15.96:0/3907137959 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f71400a2e70 con 0x55d6cef6d2f0 2024-04-25T17:11:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f71533fb740 10 client.25091 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:11:49.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f71533fb740 10 client.25091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:11:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.200+0000 7f71533fb740 20 client.25091 populate_metadata read hostname 'smithi096' 2024-04-25T17:11:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.201+0000 7f71533fb740 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cf3bac40 0x55d6cf3bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:11:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.201+0000 7f71533fb740 4 client.25091 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:11:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.201+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7134005850 con 0x55d6cf3bac40 2024-04-25T17:11:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.201+0000 7f7149a02640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cf3bac40 0x55d6cf3bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:11:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.201+0000 7f7149a02640 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cf3bac40 0x55d6cf3bd080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=840127d23fb84ba6 server_cookie=7244dfeb467ffa08 in_seq=0 out_seq=0 2024-04-25T17:11:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.201+0000 7f712dffb640 10 client.25091 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f712dffb640 1 -- 172.21.15.96:0/3907137959 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7134005850 con 0x55d6cf3bac40 2024-04-25T17:11:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f712dffb640 10 client.25091 handle_command_reply: tid=0 2024-04-25T17:11:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 client.25091 shutdown 2024-04-25T17:11:49.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef6d2f0 msgr2=0x55d6cf355890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef6d2f0 0x55d6cf355890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71400dca00 tx=0x7f7140009740 comp rx=0 tx=0).stop 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 shutdown_connections 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d6cf3bac40 0x55d6cf3bd080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6cef2a1c0 0x55d6cf3532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d6cef961b0 0x55d6cf352c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f712dffb640 0 client.25091 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 --2- 172.21.15.96:0/3907137959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d6cef6d2f0 0x55d6cf355890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 >> 172.21.15.96:0/3907137959 conn(0x55d6cef8a300 msgr2=0x55d6cf3414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 shutdown_connections 2024-04-25T17:11:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 1 -- 172.21.15.96:0/3907137959 wait complete. 2024-04-25T17:11:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.207+0000 7f71533fb740 20 client.25091 trim_cache size 0 max 16384 2024-04-25T17:11:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:11:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:11:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:11:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f71300afc50 msgr2=0x7f71300b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f71300afc50 0x7f71300b2160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55d6cf332c60 tx=0x7f713c011040 comp rx=0 tx=0).stop 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71441031c0 msgr2=0x7f71441996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71441031c0 0x7f71441996b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f714000e9f0 tx=0x7f714000eeb0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 shutdown_connections 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f71300afc50 0x7f71300b2160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7144108f00 0x7f714419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7144108a60 0x7f7144199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 --2- 172.21.15.96:0/2711873395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71441031c0 0x7f71441996b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:11:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 >> 172.21.15.96:0/2711873395 conn(0x7f71440781b0 msgr2=0x7f7144104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:11:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 shutdown_connections 2024-04-25T17:11:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:11:49.208+0000 7f714c467640 1 -- 172.21.15.96:0/2711873395 wait complete. 2024-04-25T17:11:50.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:50 smithi096 ceph-mon[29131]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:50.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:50 smithi097 ceph-mon[32602]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:50.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:50 smithi171 ceph-mon[33189]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:52.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:52 smithi096 ceph-mon[29131]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-25T17:11:52.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:52 smithi097 ceph-mon[32602]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-25T17:11:52.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:52 smithi171 ceph-mon[33189]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-25T17:11:54.689 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:54 smithi096 ceph-mon[29131]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:54.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:54 smithi097 ceph-mon[32602]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:54.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:54 smithi171 ceph-mon[33189]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:56.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:56 smithi096 ceph-mon[29131]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:56.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:56 smithi097 ceph-mon[32602]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:56.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:56 smithi171 ceph-mon[33189]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr: git switch -c 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T17:11:57.820 INFO:tasks.workunit.client.0.smithi096.stderr:Or undo this operation with: 2024-04-25T17:11:57.821 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T17:11:57.821 INFO:tasks.workunit.client.0.smithi096.stderr: git switch - 2024-04-25T17:11:57.821 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T17:11:57.821 INFO:tasks.workunit.client.0.smithi096.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T17:11:57.821 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T17:11:57.821 INFO:tasks.workunit.client.0.smithi096.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T17:11:57.826 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T17:11:57.961 INFO:tasks.workunit.client.0.smithi096.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T17:11:57.963 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T17:11:57.963 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T17:11:58.610 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T17:11:58.665 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T17:11:58.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:11:58 smithi096 ceph-mon[29131]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:58.704 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T17:11:58.706 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T17:11:58.706 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T17:11:58.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:11:58 smithi097 ceph-mon[32602]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:58.744 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T17:11:58.747 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:11:58.747 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T17:11:58.763 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-25T17:11:58.764 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-25T17:11:58.764 DEBUG:teuthology.orchestra.run.smithi096:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-25T17:11:58.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:11:58 smithi171 ceph-mon[33189]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T17:11:58.823 INFO:tasks.workunit.client.0.smithi096.stderr:+ set -e 2024-04-25T17:11:58.824 INFO:tasks.workunit.client.0.smithi096.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-25T17:11:58.825 INFO:tasks.workunit.client.0.smithi096.stderr:Cloning into 'xfstests-dev'... 2024-04-25T17:12:00.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:00 smithi096 ceph-mon[29131]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:12:00.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:00 smithi097 ceph-mon[32602]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:12:00.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:00 smithi171 ceph-mon[33189]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T17:12:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 -- 172.21.15.96:0/1579818430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 msgr2=0x7fde1c101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 --2- 172.21.15.96:0/1579818430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c101c20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fde18009ca0 tx=0x7fde1802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 -- 172.21.15.96:0/1579818430 shutdown_connections 2024-04-25T17:12:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 --2- 172.21.15.96:0/1579818430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c102810 0x7fde1c110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 --2- 172.21.15.96:0/1579818430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde1c10a010 0x7fde1c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 --2- 172.21.15.96:0/1579818430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c101c20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 -- 172.21.15.96:0/1579818430 >> 172.21.15.96:0/1579818430 conn(0x7fde1c0778b0 msgr2=0x7fde1c077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 -- 172.21.15.96:0/1579818430 shutdown_connections 2024-04-25T17:12:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.319+0000 7fde2389f640 1 -- 172.21.15.96:0/1579818430 wait complete. 2024-04-25T17:12:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 Processor -- start 2024-04-25T17:12:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 -- start start 2024-04-25T17:12:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde1c19a4c0 0x7fde1c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c19ece0 0x7fde1c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c113d60 con 0x7fde1c109b70 2024-04-25T17:12:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c113be0 con 0x7fde1c19ece0 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2389f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c113ee0 con 0x7fde1c19a4c0 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55058/0 (socket says 172.21.15.96:55058) 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 -- 172.21.15.96:0/593069743 learned_addr learned my addr 172.21.15.96:0/593069743 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde21e3c640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c19ece0 0x7fde1c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde20e3a640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde1c19a4c0 0x7fde1c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 -- 172.21.15.96:0/593069743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde1c19a4c0 msgr2=0x7fde1c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde1c19a4c0 0x7fde1c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 -- 172.21.15.96:0/593069743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c19ece0 msgr2=0x7fde1c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c19ece0 0x7fde1c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde2163b640 1 -- 172.21.15.96:0/593069743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c1a6a50 con 0x7fde1c109b70 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.320+0000 7fde20e3a640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde1c19a4c0 0x7fde1c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.321+0000 7fde2163b640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c199f80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde18006010 tx=0x7fde180319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.321+0000 7fde127fc640 1 -- 172.21.15.96:0/593069743 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18003bf0 con 0x7fde1c109b70 2024-04-25T17:12:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.321+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c1a6c20 con 0x7fde1c109b70 2024-04-25T17:12:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.321+0000 7fde127fc640 1 -- 172.21.15.96:0/593069743 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde18003d90 con 0x7fde1c109b70 2024-04-25T17:12:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.321+0000 7fde127fc640 1 -- 172.21.15.96:0/593069743 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1802fb60 con 0x7fde1c109b70 2024-04-25T17:12:02.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.321+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1a7080 con 0x7fde1c109b70 2024-04-25T17:12:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.323+0000 7fde127fc640 1 -- 172.21.15.96:0/593069743 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fde1802fd00 con 0x7fde1c109b70 2024-04-25T17:12:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 Processor -- start 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 -- start start 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde127fc640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fde080afc50 0x7fde080b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac411b0 0x55d4bafe0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac182f0 0x55d4bafe1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d4baf47460 con 0x55d4bac182f0 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d4baf01590 con 0x55d4bac411b0 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2a82b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d4baf43420 con 0x55d4bac204d0 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde21e3c640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fde080afc50 0x7fde080b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde20e3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac182f0 0x55d4bafe1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde20e3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac182f0 0x55d4bafe1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55072/0 (socket says 172.21.15.96:55072) 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde20e3a640 1 -- 172.21.15.96:0/805718348 learned_addr learned my addr 172.21.15.96:0/805718348 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde127fc640 1 -- 172.21.15.96:0/593069743 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7fde18100560 con 0x7fde1c109b70 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde21e3c640 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.324+0000 7fde2163b640 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac411b0 0x55d4bafe0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde20e3a640 1 -- 172.21.15.96:0/805718348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 msgr2=0x55d4bafe1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde20e3a640 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde20e3a640 1 -- 172.21.15.96:0/805718348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac411b0 msgr2=0x55d4bafe0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde20e3a640 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac411b0 0x55d4bafe0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde20e3a640 1 -- 172.21.15.96:0/805718348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4baf0e0d0 con 0x55d4bac182f0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde21e3c640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fde080afc50 0x7fde080b2160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde14005fd0 tx=0x7fde140137d0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde20e3a640 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac182f0 0x55d4bafe1030 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ba70 tx=0x7fde0c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708636f234eb739e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde077fe640 1 -- 172.21.15.96:0/805718348 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c017420 con 0x55d4bac182f0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde077fe640 1 -- 172.21.15.96:0/805718348 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0c017a70 con 0x55d4bac182f0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.325+0000 7fde077fe640 1 -- 172.21.15.96:0/805718348 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00f9c0 con 0x55d4bac182f0 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 -- 172.21.15.96:0/805718348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac182f0 msgr2=0x55d4bafe1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac182f0 0x55d4bafe1030 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ba70 tx=0x7fde0c00bf30 comp rx=0 tx=0).stop 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 -- 172.21.15.96:0/805718348 shutdown_connections 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac182f0 0x55d4bafe1030 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 --2- 172.21.15.96:0/805718348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac411b0 0x55d4bafe0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 -- 172.21.15.96:0/805718348 >> 172.21.15.96:0/805718348 conn(0x55d4bac35300 msgr2=0x55d4bafdd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 -- 172.21.15.96:0/805718348 shutdown_connections 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 -- 172.21.15.96:0/805718348 wait complete. 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 -- start start 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.326+0000 7fde2a82b740 1 Processor -- start 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bafeb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafeb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac411b0 0x55d4baff70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2a82b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d4bafedc20 con 0x55d4bac411b0 2024-04-25T17:12:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2a82b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d4baeceb60 con 0x55d4bac182f0 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2a82b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d4bafedda0 con 0x55d4bac204d0 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2a82b740 10 client.0 _ll_register_callbacks cb 0x55d4baef32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac411b0 0x55d4baff70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac411b0 0x55d4baff70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55084/0 (socket says 172.21.15.96:55084) 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 -- 172.21.15.96:0/249367343 learned_addr learned my addr 172.21.15.96:0/249367343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde2163b640 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bafeb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde20e3a640 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafeb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 -- 172.21.15.96:0/249367343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 msgr2=0x55d4bafeb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafeb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 -- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 msgr2=0x55d4bafeb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bafeb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.327+0000 7fde21e3c640 1 -- 172.21.15.96:0/249367343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bac59210 con 0x55d4bac411b0 2024-04-25T17:12:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde21e3c640 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac411b0 0x55d4baff70d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde14011990 tx=0x7fde14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9befb257bb8f00ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde04ff9640 1 -- 172.21.15.96:0/249367343 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14025070 con 0x55d4bac411b0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde04ff9640 1 -- 172.21.15.96:0/249367343 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7fde1400fd00 con 0x55d4bac411b0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde0c017a70 con 0x55d4bac411b0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde04ff9640 1 -- 172.21.15.96:0/249367343 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fde14031050 con 0x55d4bac411b0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 10 client.15489 fetch_fsmap learned FSMap version 18 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d4bad25620 con 0x55d4bac411b0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde04ff9640 1 -- 172.21.15.96:0/249367343 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0c017a70 con 0x55d4bac411b0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde04ff9640 1 -- 172.21.15.96:0/249367343 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fde1400f050 con 0x55d4bac411b0 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 10 client.15489 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 20 client.15489 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4babffd80 0x55d4baffd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 4 client.15489 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:12:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.328+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4baf55070 con 0x55d4babffd80 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.329+0000 7fde2163b640 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4babffd80 0x55d4baffd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.329+0000 7fde2163b640 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4babffd80 0x55d4baffd660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61eb1f629f0826a8 server_cookie=4c4f597529b80fb1 in_seq=0 out_seq=0 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.329+0000 7fde04ff9640 10 client.15489 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde04ff9640 1 -- 172.21.15.96:0/249367343 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d4baf55070 con 0x55d4babffd80 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde04ff9640 10 client.15489 handle_command_reply: tid=0 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 client.15489 shutdown 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac411b0 msgr2=0x55d4baff70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac411b0 0x55d4baff70d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde14011990 tx=0x7fde14019000 comp rx=0 tx=0).stop 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 shutdown_connections 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4babffd80 0x55d4baffd660 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4bac411b0 0x55d4baff70d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafeb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde04ff9640 0 client.15489 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 --2- 172.21.15.96:0/249367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bafeb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 >> 172.21.15.96:0/249367343 conn(0x55d4bac35300 msgr2=0x55d4bafe0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 shutdown_connections 2024-04-25T17:12:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 1 -- 172.21.15.96:0/249367343 wait complete. 2024-04-25T17:12:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.330+0000 7fde2a82b740 20 client.15489 trim_cache size 0 max 16384 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.345+0000 7fde2a82b740 1 Processor -- start 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.345+0000 7fde2a82b740 1 -- start start 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4bb002e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bb0033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2a82b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1ad960 con 0x55d4babffd80 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2a82b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1b01e0 con 0x55d4bac182f0 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2a82b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1adfd0 con 0x55d4bac204d0 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4bb002e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4bb002e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55096/0 (socket says 172.21.15.96:55096) 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 -- 172.21.15.96:0/1018622067 learned_addr learned my addr 172.21.15.96:0/1018622067 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde20e3a640 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bb0033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde21e3c640 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 -- 172.21.15.96:0/1018622067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 msgr2=0x55d4bafe0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 -- 172.21.15.96:0/1018622067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 msgr2=0x55d4bb0033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bb0033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 -- 172.21.15.96:0/1018622067 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bafe9440 con 0x55d4babffd80 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde2163b640 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4bb002e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde180c4850 tx=0x7fde1802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563b362639d4b9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde06ffd640 1 -- 172.21.15.96:0/1018622067 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180ccbd0 con 0x55d4babffd80 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde06ffd640 1 -- 172.21.15.96:0/1018622067 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde18036040 con 0x55d4babffd80 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.346+0000 7fde06ffd640 1 -- 172.21.15.96:0/1018622067 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18004680 con 0x55d4babffd80 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 -- 172.21.15.96:0/1018622067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 msgr2=0x55d4bb002e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4bb002e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde180c4850 tx=0x7fde1802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 -- 172.21.15.96:0/1018622067 shutdown_connections 2024-04-25T17:12:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac204d0 0x55d4bafe0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac182f0 0x55d4bb0033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1018622067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4bb002e60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 -- 172.21.15.96:0/1018622067 >> 172.21.15.96:0/1018622067 conn(0x55d4bac35300 msgr2=0x55d4bafe97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 -- 172.21.15.96:0/1018622067 shutdown_connections 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.347+0000 7fde2a82b740 1 -- 172.21.15.96:0/1018622067 wait complete. 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 -- start start 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 Processor -- start 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4baff3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac182f0 0x55d4baff0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac204d0 0x55d4baff10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1ae6a0 con 0x55d4babffd80 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1afb10 con 0x55d4bac204d0 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1aed70 con 0x55d4bac182f0 2024-04-25T17:12:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2a82b740 10 client.0 _ll_register_callbacks cb 0x7fde1400b4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2163b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4baff3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2163b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4baff3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55102/0 (socket says 172.21.15.96:55102) 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde2163b640 1 -- 172.21.15.96:0/1233627362 learned_addr learned my addr 172.21.15.96:0/1233627362 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde20e3a640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac182f0 0x55d4baff0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.348+0000 7fde21e3c640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac204d0 0x55d4baff10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2163b640 1 -- 172.21.15.96:0/1233627362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac182f0 msgr2=0x55d4baff0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2163b640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac182f0 0x55d4baff0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2163b640 1 -- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac204d0 msgr2=0x55d4baff10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2163b640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac204d0 0x55d4baff10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2163b640 1 -- 172.21.15.96:0/1233627362 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4baff1830 con 0x55d4babffd80 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde21e3c640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac204d0 0x55d4baff10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2163b640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4baff3700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde180c8520 tx=0x7fde180c88e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0355ea07edce1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde04ff9640 1 -- 172.21.15.96:0/1233627362 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180fdf40 con 0x55d4babffd80 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde04ff9640 1 -- 172.21.15.96:0/1233627362 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7fde18004680 con 0x55d4babffd80 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde18036040 con 0x55d4babffd80 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde04ff9640 1 -- 172.21.15.96:0/1233627362 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fde180ccbd0 con 0x55d4babffd80 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 10 client.15495 fetch_fsmap learned FSMap version 18 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55d4bb065a70 con 0x55d4babffd80 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde04ff9640 1 -- 172.21.15.96:0/1233627362 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde18036040 con 0x55d4babffd80 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde04ff9640 1 -- 172.21.15.96:0/1233627362 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7fde18094270 con 0x55d4babffd80 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 10 client.15495 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 20 client.15495 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4bac411b0 0x55d4bb0680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 4 client.15495 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:12:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.349+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4baf4eee0 con 0x55d4bac411b0 2024-04-25T17:12:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.350+0000 7fde20e3a640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4bac411b0 0x55d4bb0680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.350+0000 7fde04ff9640 10 client.15495 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.350+0000 7fde20e3a640 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4bac411b0 0x55d4bb0680e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140586a81d813ac0 server_cookie=4d18cd73fa4f1536 in_seq=0 out_seq=0 2024-04-25T17:12:02.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.606+0000 7fde04ff9640 1 -- 172.21.15.96:0/1233627362 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4baf4eee0 con 0x55d4bac411b0 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.606+0000 7fde04ff9640 10 client.15495 handle_command_reply: tid=0 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.606+0000 7fde2a82b740 1 client.15495 shutdown 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 msgr2=0x55d4baff3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4baff3700 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde180c8520 tx=0x7fde180c88e0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 shutdown_connections 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55d4bac411b0 0x55d4bb0680e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4bac204d0 0x55d4baff10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d4bac182f0 0x55d4baff0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde04ff9640 0 client.15495 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 --2- 172.21.15.96:0/1233627362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4babffd80 0x55d4baff3700 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 >> 172.21.15.96:0/1233627362 conn(0x55d4bac35300 msgr2=0x55d4bafe0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:02.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 shutdown_connections 2024-04-25T17:12:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 1 -- 172.21.15.96:0/1233627362 wait complete. 2024-04-25T17:12:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2a82b740 20 client.15495 trim_cache size 0 max 16384 2024-04-25T17:12:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:12:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:12:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:12:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:12:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fde080afc50 msgr2=0x7fde080b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fde080afc50 0x7fde080b2160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fde14005fd0 tx=0x7fde140137d0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 msgr2=0x7fde1c199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c199f80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde18006010 tx=0x7fde180319f0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 shutdown_connections 2024-04-25T17:12:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fde080afc50 0x7fde080b2160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde1c19ece0 0x7fde1c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde1c19a4c0 0x7fde1c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.607+0000 7fde2389f640 1 --2- 172.21.15.96:0/593069743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde1c109b70 0x7fde1c199f80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.608+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 >> 172.21.15.96:0/593069743 conn(0x7fde1c0778b0 msgr2=0x7fde1c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.608+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 shutdown_connections 2024-04-25T17:12:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:02.608+0000 7fde2389f640 1 -- 172.21.15.96:0/593069743 wait complete. 2024-04-25T17:12:02.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:02 smithi096 ceph-mon[29131]: pgmap v56: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-25T17:12:02.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:02 smithi097 ceph-mon[32602]: pgmap v56: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-25T17:12:02.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:02 smithi171 ceph-mon[33189]: pgmap v56: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.1 MiB/s wr, 9 op/s 2024-04-25T17:12:04.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:04 smithi096 ceph-mon[29131]: pgmap v57: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T17:12:04.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:04 smithi097 ceph-mon[32602]: pgmap v57: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T17:12:04.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:04 smithi171 ceph-mon[33189]: pgmap v57: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T17:12:06.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:06 smithi096 ceph-mon[29131]: pgmap v58: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T17:12:06.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:06 smithi097 ceph-mon[32602]: pgmap v58: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T17:12:06.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:06 smithi171 ceph-mon[33189]: pgmap v58: 97 pgs: 97 active+clean; 14 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 MiB/s wr, 8 op/s 2024-04-25T17:12:08.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:08 smithi096 ceph-mon[29131]: pgmap v59: 97 pgs: 97 active+clean; 16 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-04-25T17:12:08.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:08 smithi097 ceph-mon[32602]: pgmap v59: 97 pgs: 97 active+clean; 16 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-04-25T17:12:08.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:08 smithi171 ceph-mon[33189]: pgmap v59: 97 pgs: 97 active+clean; 16 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 92 op/s 2024-04-25T17:12:10.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:10 smithi097 ceph-mon[32602]: pgmap v60: 97 pgs: 97 active+clean; 16 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-04-25T17:12:10.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:10 smithi171 ceph-mon[33189]: pgmap v60: 97 pgs: 97 active+clean; 16 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-04-25T17:12:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:10 smithi096 ceph-mon[29131]: pgmap v60: 97 pgs: 97 active+clean; 16 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 91 op/s 2024-04-25T17:12:12.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:12 smithi097 ceph-mon[32602]: pgmap v61: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2024-04-25T17:12:12.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:12 smithi171 ceph-mon[33189]: pgmap v61: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2024-04-25T17:12:12.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:12 smithi096 ceph-mon[29131]: pgmap v61: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2024-04-25T17:12:14.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:14 smithi097 ceph-mon[32602]: pgmap v62: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 303 KiB/s wr, 86 op/s 2024-04-25T17:12:14.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:14 smithi171 ceph-mon[33189]: pgmap v62: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 303 KiB/s wr, 86 op/s 2024-04-25T17:12:14.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:14 smithi096 ceph-mon[29131]: pgmap v62: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 303 KiB/s wr, 86 op/s 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 -- 172.21.15.96:0/1189117151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b01031b0 msgr2=0x7f03b010f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/1189117151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b01031b0 0x7f03b010f4e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ca0 tx=0x7f03a802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 -- 172.21.15.96:0/1189117151 shutdown_connections 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/1189117151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b01031b0 0x7f03b010f4e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/1189117151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03b0108ef0 0x7f03b0102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/1189117151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0108a50 0x7f03b0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 -- 172.21.15.96:0/1189117151 >> 172.21.15.96:0/1189117151 conn(0x7f03b00fe260 msgr2=0x7f03b0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 -- 172.21.15.96:0/1189117151 shutdown_connections 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.712+0000 7f03b5dc5640 1 -- 172.21.15.96:0/1189117151 wait complete. 2024-04-25T17:12:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 Processor -- start 2024-04-25T17:12:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 -- start start 2024-04-25T17:12:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03b01031b0 0x7f03b019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0108a50 0x7f03b019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b0108ef0 0x7f03b019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0112c90 con 0x7f03b0108ef0 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0112b10 con 0x7f03b0108a50 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03b5dc5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0112e10 con 0x7f03b01031b0 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b0108ef0 0x7f03b019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b0108ef0 0x7f03b019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60762/0 (socket says 172.21.15.96:60762) 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03affff640 1 -- 172.21.15.96:0/3645002242 learned_addr learned my addr 172.21.15.96:0/3645002242 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.713+0000 7f03aeffd640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0108a50 0x7f03b019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03af7fe640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03b01031b0 0x7f03b019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03affff640 1 -- 172.21.15.96:0/3645002242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03b01031b0 msgr2=0x7f03b019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03affff640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03b01031b0 0x7f03b019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03affff640 1 -- 172.21.15.96:0/3645002242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0108a50 msgr2=0x7f03b019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03affff640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0108a50 0x7f03b019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03affff640 1 -- 172.21.15.96:0/3645002242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b019f5a0 con 0x7f03b0108ef0 2024-04-25T17:12:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03affff640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b0108ef0 0x7f03b019ef60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098d0 tx=0x7f03a80317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03acff9640 1 -- 172.21.15.96:0/3645002242 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8003b70 con 0x7f03b0108ef0 2024-04-25T17:12:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b019f830 con 0x7f03b0108ef0 2024-04-25T17:12:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03acff9640 1 -- 172.21.15.96:0/3645002242 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a8003d10 con 0x7f03b0108ef0 2024-04-25T17:12:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03acff9640 1 -- 172.21.15.96:0/3645002242 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a802fa20 con 0x7f03b0108ef0 2024-04-25T17:12:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.714+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b01a7210 con 0x7f03b0108ef0 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 Processor -- start 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03acff9640 1 -- 172.21.15.96:0/3645002242 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f03a802fc00 con 0x7f03b0108ef0 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 -- start start 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74fa51b0 0x55cd75344af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75345030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f844d0 0x55cd75345570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cd752ab460 con 0x55cd74f844d0 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd75265590 con 0x55cd74f7c2f0 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03bcc78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55cd752a7420 con 0x55cd74fa51b0 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f844d0 0x55cd75345570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f844d0 0x55cd75345570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60774/0 (socket says 172.21.15.96:60774) 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03aeffd640 1 -- 172.21.15.96:0/2546009848 learned_addr learned my addr 172.21.15.96:0/2546009848 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03af7fe640 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75345030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.717+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74fa51b0 0x55cd75344af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03acff9640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f03980afc50 0x7f03980b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03aeffd640 1 -- 172.21.15.96:0/2546009848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74fa51b0 msgr2=0x55cd75344af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74fa51b0 0x55cd75344af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03aeffd640 1 -- 172.21.15.96:0/2546009848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 msgr2=0x55cd75345030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75345030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03aeffd640 1 -- 172.21.15.96:0/2546009848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd752720d0 con 0x55cd74f844d0 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03affff640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f03980afc50 0x7f03980b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03af7fe640 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75345030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03acff9640 1 -- 172.21.15.96:0/3645002242 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f03a81005e0 con 0x7f03b0108ef0 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f844d0 0x55cd75345570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55cd75341c60 tx=0x7f03a4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7a369651464c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f0395ffb640 1 -- 172.21.15.96:0/2546009848 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4012ca0 con 0x55cd74f844d0 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f0395ffb640 1 -- 172.21.15.96:0/2546009848 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a4012e40 con 0x55cd74f844d0 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.718+0000 7f0395ffb640 1 -- 172.21.15.96:0/2546009848 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4010740 con 0x55cd74f844d0 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 -- 172.21.15.96:0/2546009848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f844d0 msgr2=0x55cd75345570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f844d0 0x55cd75345570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55cd75341c60 tx=0x7f03a4005c90 comp rx=0 tx=0).stop 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03affff640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f03980afc50 0x7f03980b2160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03a80fdf00 tx=0x7f03a80973a0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 -- 172.21.15.96:0/2546009848 shutdown_connections 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f844d0 0x55cd75345570 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75345030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2546009848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74fa51b0 0x55cd75344af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 -- 172.21.15.96:0/2546009848 >> 172.21.15.96:0/2546009848 conn(0x55cd74f99300 msgr2=0x55cd753412a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 -- 172.21.15.96:0/2546009848 shutdown_connections 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 -- 172.21.15.96:0/2546009848 wait complete. 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.719+0000 7f03bcc78740 1 -- start start 2024-04-25T17:12:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 1 Processor -- start 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f7c2f0 0x55cd75350c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd75351190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74fa51b0 0x55cd7534f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cd75351c20 con 0x55cd74f7c2f0 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cd75232b60 con 0x55cd74fa51b0 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55cd75351da0 con 0x55cd74f844d0 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03bcc78740 10 client.0 _ll_register_callbacks cb 0x55cd752572b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f7c2f0 0x55cd75350c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f7c2f0 0x55cd75350c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60778/0 (socket says 172.21.15.96:60778) 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 -- 172.21.15.96:0/521734604 learned_addr learned my addr 172.21.15.96:0/521734604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03af7fe640 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd75351190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74fa51b0 0x55cd7534f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 -- 172.21.15.96:0/521734604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 msgr2=0x55cd75351190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd75351190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 -- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74fa51b0 msgr2=0x55cd7534f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74fa51b0 0x55cd7534f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.720+0000 7f03aeffd640 1 -- 172.21.15.96:0/521734604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd74fbd210 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03aeffd640 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f7c2f0 0x55cd75350c50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f03a4004bf0 tx=0x7f03a4005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef75ee7c915febb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f038effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f038effd640 1 -- 172.21.15.96:0/521734604 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4004fd0 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f038effd640 1 -- 172.21.15.96:0/521734604 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f03a4005170 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd752b9070 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f038effd640 1 -- 172.21.15.96:0/521734604 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f03a4005370 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 10 client.15507 fetch_fsmap learned FSMap version 18 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cd75089620 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f038effd640 1 -- 172.21.15.96:0/521734604 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cd752b9070 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f038effd640 1 -- 172.21.15.96:0/521734604 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f03a402d4b0 con 0x55cd74f7c2f0 2024-04-25T17:12:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 10 client.15507 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 20 client.15507 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f63d80 0x55cd753616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 4 client.15507 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.721+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd752b2ee0 con 0x55cd74f63d80 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.722+0000 7f03af7fe640 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f63d80 0x55cd753616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.722+0000 7f038effd640 10 client.15507 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.722+0000 7f03af7fe640 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f63d80 0x55cd753616b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198c04d308f1b9d9 server_cookie=9dacacabf0485aff in_seq=0 out_seq=0 2024-04-25T17:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f038effd640 1 -- 172.21.15.96:0/521734604 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd752b2ee0 con 0x55cd74f63d80 2024-04-25T17:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f038effd640 10 client.15507 handle_command_reply: tid=0 2024-04-25T17:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 client.15507 shutdown 2024-04-25T17:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f7c2f0 msgr2=0x55cd75350c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f7c2f0 0x55cd75350c50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f03a4004bf0 tx=0x7f03a4005f30 comp rx=0 tx=0).stop 2024-04-25T17:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 shutdown_connections 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f63d80 0x55cd753616b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74fa51b0 0x55cd7534f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd75351190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 --2- 172.21.15.96:0/521734604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74f7c2f0 0x55cd75350c50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f038effd640 0 client.15507 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.723+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 >> 172.21.15.96:0/521734604 conn(0x55cd74f99300 msgr2=0x55cd75346810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.724+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 shutdown_connections 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.724+0000 7f03bcc78740 1 -- 172.21.15.96:0/521734604 wait complete. 2024-04-25T17:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.724+0000 7f03bcc78740 20 client.15507 trim_cache size 0 max 16384 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.738+0000 7f03bcc78740 1 Processor -- start 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.738+0000 7f03bcc78740 1 -- start start 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75350f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7534c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd7534c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03bcc78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01ad9f0 con 0x55cd74fa51b0 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03bcc78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01b0270 con 0x55cd74f7c2f0 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03bcc78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01ae060 con 0x55cd74f844d0 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd7534c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd7534c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60788/0 (socket says 172.21.15.96:60788) 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 -- 172.21.15.96:0/2243062625 learned_addr learned my addr 172.21.15.96:0/2243062625 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75350f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 -- 172.21.15.96:0/2243062625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 msgr2=0x55cd7534c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7534c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 -- 172.21.15.96:0/2243062625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 msgr2=0x55cd75350f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75350f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 -- 172.21.15.96:0/2243062625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7534d620 con 0x55cd74fa51b0 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03aeffd640 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd7534c850 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b910 tx=0x7f03a4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437d2cb2100f4d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03957fa640 1 -- 172.21.15.96:0/2243062625 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4039070 con 0x55cd74fa51b0 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03957fa640 1 -- 172.21.15.96:0/2243062625 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a4016070 con 0x55cd74fa51b0 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.739+0000 7f03957fa640 1 -- 172.21.15.96:0/2243062625 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a402d4f0 con 0x55cd74fa51b0 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 -- 172.21.15.96:0/2243062625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 msgr2=0x55cd7534c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd7534c850 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b910 tx=0x7f03a4014040 comp rx=0 tx=0).stop 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 -- 172.21.15.96:0/2243062625 shutdown_connections 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd7534c850 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7534c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 --2- 172.21.15.96:0/2243062625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd75350f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 -- 172.21.15.96:0/2243062625 >> 172.21.15.96:0/2243062625 conn(0x55cd74f99300 msgr2=0x55cd75350860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:15.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 -- 172.21.15.96:0/2243062625 shutdown_connections 2024-04-25T17:12:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 -- 172.21.15.96:0/2243062625 wait complete. 2024-04-25T17:12:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.740+0000 7f03bcc78740 1 -- start start 2024-04-25T17:12:15.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 1 Processor -- start 2024-04-25T17:12:15.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd7535a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7535aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd753c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01ae730 con 0x55cd74fa51b0 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01afba0 con 0x55cd74f7c2f0 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03b01aee00 con 0x55cd74f844d0 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03bcc78740 10 client.0 _ll_register_callbacks cb 0x7f03a00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd753c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd753c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60790/0 (socket says 172.21.15.96:60790) 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 -- 172.21.15.96:0/4151365201 learned_addr learned my addr 172.21.15.96:0/4151365201 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd7535a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03af7fe640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7535aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 -- 172.21.15.96:0/4151365201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 msgr2=0x55cd7535aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7535aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 -- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 msgr2=0x55cd7535a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd7535a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03aeffd640 1 -- 172.21.15.96:0/4151365201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7535b2b0 con 0x55cd74fa51b0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.741+0000 7f03af7fe640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7535aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03aeffd640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd753c77e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03a4006010 tx=0x7f03a4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdb339f82c7e9db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f038effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f038effd640 1 -- 172.21.15.96:0/4151365201 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4039040 con 0x55cd74fa51b0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a4016070 con 0x55cd74fa51b0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f038effd640 1 -- 172.21.15.96:0/4151365201 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f03a402d240 con 0x55cd74fa51b0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f038effd640 1 -- 172.21.15.96:0/4151365201 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f03a400ba60 con 0x55cd74fa51b0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 10 client.15513 fetch_fsmap learned FSMap version 18 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x55cd753c8d10 con 0x55cd74fa51b0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f038effd640 1 -- 172.21.15.96:0/4151365201 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a4016070 con 0x55cd74fa51b0 2024-04-25T17:12:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f038effd640 1 -- 172.21.15.96:0/4151365201 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f03a402d4d0 con 0x55cd74fa51b0 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 10 client.15513 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 20 client.15513 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f391c0 0x55cd753cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 4 client.15513 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.742+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd7522e5f0 con 0x55cd74f391c0 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.743+0000 7f03af7fe640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f391c0 0x55cd753cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.743+0000 7f038effd640 10 client.15513 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:15.743+0000 7f03af7fe640 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f391c0 0x55cd753cb2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116c5d93fd361c4e server_cookie=80b99f7fb248a8f0 in_seq=0 out_seq=0 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f038effd640 1 -- 172.21.15.96:0/4151365201 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd7522e5f0 con 0x55cd74f391c0 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f038effd640 10 client.15513 handle_command_reply: tid=0 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 client.15513 shutdown 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 msgr2=0x55cd753c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd753c77e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03a4006010 tx=0x7f03a4005c90 comp rx=0 tx=0).stop 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 shutdown_connections 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55cd74f391c0 0x55cd753cb2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cd74fa51b0 0x55cd753c77e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cd74f844d0 0x55cd7535aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f038effd640 0 client.15513 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 --2- 172.21.15.96:0/4151365201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cd74f7c2f0 0x55cd7535a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 >> 172.21.15.96:0/4151365201 conn(0x55cd74f99300 msgr2=0x55cd75344970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 shutdown_connections 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 1 -- 172.21.15.96:0/4151365201 wait complete. 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.283+0000 7f03bcc78740 20 client.15513 trim_cache size 0 max 16384 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:12:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:12:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f03980afc50 msgr2=0x7f03980b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f03980afc50 0x7f03980b2160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03a80fdf00 tx=0x7f03a80973a0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b0108ef0 msgr2=0x7f03b019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b0108ef0 0x7f03b019ef60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098d0 tx=0x7f03a80317b0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 shutdown_connections 2024-04-25T17:12:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03b0108ef0 0x7f03b019ef60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f03980afc50 0x7f03980b2160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03b0108a50 0x7f03b019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 --2- 172.21.15.96:0/3645002242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03b01031b0 0x7f03b019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 >> 172.21.15.96:0/3645002242 conn(0x7f03b00fe260 msgr2=0x7f03b01066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 shutdown_connections 2024-04-25T17:12:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:16.284+0000 7f03b5dc5640 1 -- 172.21.15.96:0/3645002242 wait complete. 2024-04-25T17:12:16.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:16 smithi097 ceph-mon[32602]: pgmap v63: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 303 KiB/s wr, 86 op/s 2024-04-25T17:12:16.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:16 smithi171 ceph-mon[33189]: pgmap v63: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 303 KiB/s wr, 86 op/s 2024-04-25T17:12:16.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:16 smithi096 ceph-mon[29131]: pgmap v63: 97 pgs: 97 active+clean; 17 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 303 KiB/s wr, 86 op/s 2024-04-25T17:12:18.145 INFO:tasks.workunit.client.0.smithi096.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 11% (490/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 28% (1203/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 40% (1726/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 52% (2259/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 85% (3651/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 97% (4194/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-25T17:12:18.151 INFO:tasks.workunit.client.0.smithi096.stderr:+ cd xfstests-dev 2024-04-25T17:12:18.151 INFO:tasks.workunit.client.0.smithi096.stderr:+ make -j4 2024-04-25T17:12:18.187 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize -cfi 2024-04-25T17:12:18.538 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T17:12:18.538 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './config.guess' 2024-04-25T17:12:18.543 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './config.sub' 2024-04-25T17:12:18.548 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './install-sh' 2024-04-25T17:12:18.553 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T17:12:18.575 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T17:12:18.575 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T17:12:18.581 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T17:12:18.587 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T17:12:18.592 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T17:12:18.598 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T17:12:18.604 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T17:12:18.605 INFO:tasks.workunit.client.0.smithi096.stdout:cp include/install-sh . 2024-04-25T17:12:18.618 INFO:tasks.workunit.client.0.smithi096.stdout:aclocal -I m4 2024-04-25T17:12:18.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:18 smithi097 ceph-mon[32602]: pgmap v64: 97 pgs: 97 active+clean; 21 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 710 KiB/s wr, 106 op/s 2024-04-25T17:12:18.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:18 smithi171 ceph-mon[33189]: pgmap v64: 97 pgs: 97 active+clean; 21 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 710 KiB/s wr, 106 op/s 2024-04-25T17:12:18.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:18 smithi096 ceph-mon[29131]: pgmap v64: 97 pgs: 97 active+clean; 21 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 710 KiB/s wr, 106 op/s 2024-04-25T17:12:19.633 INFO:tasks.workunit.client.0.smithi096.stdout:autoheader 2024-04-25T17:12:20.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:20 smithi097 ceph-mon[32602]: pgmap v65: 97 pgs: 97 active+clean; 21 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 559 KiB/s wr, 23 op/s 2024-04-25T17:12:20.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:20 smithi171 ceph-mon[33189]: pgmap v65: 97 pgs: 97 active+clean; 21 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 559 KiB/s wr, 23 op/s 2024-04-25T17:12:20.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:20 smithi096 ceph-mon[29131]: pgmap v65: 97 pgs: 97 active+clean; 21 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 559 KiB/s wr, 23 op/s 2024-04-25T17:12:21.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:21 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:12:21.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:21 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:12:21.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:21 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:12:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.2 MiB/s wr, 885 op/s 2024-04-25T17:12:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:12:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:12:22.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.2 MiB/s wr, 885 op/s 2024-04-25T17:12:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:12:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:12:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.2 MiB/s wr, 885 op/s 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:12:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:12:23.094 INFO:tasks.workunit.client.0.smithi096.stdout:autoconf 2024-04-25T17:12:23.272 INFO:tasks.workunit.client.0.smithi096.stdout:./configure \ 2024-04-25T17:12:23.272 INFO:tasks.workunit.client.0.smithi096.stdout: --libexecdir=/usr/lib \ 2024-04-25T17:12:23.272 INFO:tasks.workunit.client.0.smithi096.stdout: --exec_prefix=/var/lib 2024-04-25T17:12:23.405 INFO:tasks.workunit.client.0.smithi096.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T17:12:23.405 INFO:tasks.workunit.client.0.smithi096.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T17:12:23.406 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to print strings... printf 2024-04-25T17:12:23.406 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc... gcc 2024-04-25T17:12:23.492 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether the C compiler works... yes 2024-04-25T17:12:23.492 INFO:tasks.workunit.client.0.smithi096.stdout:checking for C compiler default output file name... a.out 2024-04-25T17:12:23.531 INFO:tasks.workunit.client.0.smithi096.stdout:checking for suffix of executables... 2024-04-25T17:12:23.569 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are cross compiling... no 2024-04-25T17:12:23.611 INFO:tasks.workunit.client.0.smithi096.stdout:checking for suffix of object files... o 2024-04-25T17:12:23.635 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T17:12:23.659 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc accepts -g... yes 2024-04-25T17:12:23.736 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T17:12:23.739 INFO:tasks.workunit.client.0.smithi096.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-25T17:12:23.744 INFO:tasks.workunit.client.0.smithi096.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T17:12:23.745 INFO:tasks.workunit.client.0.smithi096.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T17:12:23.746 INFO:tasks.workunit.client.0.smithi096.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-25T17:12:23.749 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-25T17:12:23.751 INFO:tasks.workunit.client.0.smithi096.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-25T17:12:23.757 INFO:tasks.workunit.client.0.smithi096.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-25T17:12:24.674 INFO:tasks.workunit.client.0.smithi096.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-25T17:12:24.674 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether ln -s works... yes 2024-04-25T17:12:24.692 INFO:tasks.workunit.client.0.smithi096.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T17:12:24.696 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T17:12:24.696 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T17:12:24.696 INFO:tasks.workunit.client.0.smithi096.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-25T17:12:24.696 INFO:tasks.workunit.client.0.smithi096.stdout:checking for objdump... objdump 2024-04-25T17:12:24.697 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T17:12:24.697 INFO:tasks.workunit.client.0.smithi096.stdout:checking for dlltool... no 2024-04-25T17:12:24.697 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T17:12:24.697 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ar... ar 2024-04-25T17:12:24.772 INFO:tasks.workunit.client.0.smithi096.stdout:checking for archiver @FILE support... @ 2024-04-25T17:12:24.773 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strip... strip 2024-04-25T17:12:24.773 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ranlib... ranlib 2024-04-25T17:12:24.773 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gawk... gawk 2024-04-25T17:12:24.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:24 smithi171 ceph-mon[33189]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.1 MiB/s wr, 882 op/s 2024-04-25T17:12:24.914 INFO:tasks.workunit.client.0.smithi096.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-25T17:12:24.915 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sysroot... no 2024-04-25T17:12:24.926 INFO:tasks.workunit.client.0.smithi096.stdout:checking for a working dd... /usr/bin/dd 2024-04-25T17:12:24.937 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-25T17:12:24.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:24 smithi096 ceph-mon[29131]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.1 MiB/s wr, 882 op/s 2024-04-25T17:12:24.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:24 smithi097 ceph-mon[32602]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.1 MiB/s wr, 882 op/s 2024-04-25T17:12:24.988 INFO:tasks.workunit.client.0.smithi096.stdout:checking for mt... no 2024-04-25T17:12:25.000 INFO:tasks.workunit.client.0.smithi096.stdout:checking if : is a manifest tool... no 2024-04-25T17:12:25.264 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T17:12:25.622 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ANSI C header files... yes 2024-04-25T17:12:25.687 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/types.h... yes 2024-04-25T17:12:25.732 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/stat.h... yes 2024-04-25T17:12:25.770 INFO:tasks.workunit.client.0.smithi096.stdout:checking for stdlib.h... yes 2024-04-25T17:12:25.879 INFO:tasks.workunit.client.0.smithi096.stdout:checking for string.h... yes 2024-04-25T17:12:26.033 INFO:tasks.workunit.client.0.smithi096.stdout:checking for memory.h... yes 2024-04-25T17:12:26.121 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strings.h... yes 2024-04-25T17:12:26.600 INFO:tasks.workunit.client.0.smithi096.stdout:checking for inttypes.h... yes 2024-04-25T17:12:26.651 INFO:tasks.workunit.client.0.smithi096.stdout:checking for stdint.h... yes 2024-04-25T17:12:26.705 INFO:tasks.workunit.client.0.smithi096.stdout:checking for unistd.h... yes 2024-04-25T17:12:26.770 INFO:tasks.workunit.client.0.smithi096.stdout:checking for dlfcn.h... yes 2024-04-25T17:12:26.777 INFO:tasks.workunit.client.0.smithi096.stdout:checking for objdir... .libs 2024-04-25T17:12:26.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:26 smithi171 ceph-mon[33189]: pgmap v68: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.1 MiB/s wr, 882 op/s 2024-04-25T17:12:26.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:26 smithi096 ceph-mon[29131]: pgmap v68: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.1 MiB/s wr, 882 op/s 2024-04-25T17:12:26.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:26 smithi097 ceph-mon[32602]: pgmap v68: 97 pgs: 97 active+clean; 33 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.1 MiB/s wr, 882 op/s 2024-04-25T17:12:26.990 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T17:12:26.990 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T17:12:27.027 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T17:12:27.084 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc static flag -static works... no 2024-04-25T17:12:27.140 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T17:12:27.141 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T17:12:27.146 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T17:12:27.189 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T17:12:27.257 INFO:tasks.workunit.client.0.smithi096.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T17:12:27.257 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T17:12:27.258 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether stripping libraries is possible... yes 2024-04-25T17:12:27.258 INFO:tasks.workunit.client.0.smithi096.stdout:checking if libtool supports shared libraries... yes 2024-04-25T17:12:27.258 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether to build shared libraries... yes 2024-04-25T17:12:27.258 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether to build static libraries... yes 2024-04-25T17:12:27.264 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc... (cached) gcc 2024-04-25T17:12:27.298 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T17:12:27.298 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T17:12:27.298 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T17:12:27.299 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T17:12:27.299 INFO:tasks.workunit.client.0.smithi096.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-25T17:12:27.299 INFO:tasks.workunit.client.0.smithi096.stdout:checking for tar... /bin/tar 2024-04-25T17:12:27.300 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gzip... /bin/gzip 2024-04-25T17:12:27.320 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc -MM is supported... yes 2024-04-25T17:12:27.320 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sort... /bin/sort 2024-04-25T17:12:27.320 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether ln -s works... yes 2024-04-25T17:12:27.320 INFO:tasks.workunit.client.0.smithi096.stdout:checking for rpm... /bin/rpm 2024-04-25T17:12:27.323 INFO:tasks.workunit.client.0.smithi096.stdout:checking for rpmbuild... no 2024-04-25T17:12:27.374 INFO:tasks.workunit.client.0.smithi096.stdout:checking assert.h usability... yes 2024-04-25T17:12:27.390 INFO:tasks.workunit.client.0.smithi096.stdout:checking assert.h presence... yes 2024-04-25T17:12:27.390 INFO:tasks.workunit.client.0.smithi096.stdout:checking for assert.h... yes 2024-04-25T17:12:27.440 INFO:tasks.workunit.client.0.smithi096.stdout:checking bstring.h usability... no 2024-04-25T17:12:27.468 INFO:tasks.workunit.client.0.smithi096.stdout:checking bstring.h presence... no 2024-04-25T17:12:27.468 INFO:tasks.workunit.client.0.smithi096.stdout:checking for bstring.h... no 2024-04-25T17:12:27.506 INFO:tasks.workunit.client.0.smithi096.stdout:checking libgen.h usability... yes 2024-04-25T17:12:27.522 INFO:tasks.workunit.client.0.smithi096.stdout:checking libgen.h presence... yes 2024-04-25T17:12:27.522 INFO:tasks.workunit.client.0.smithi096.stdout:checking for libgen.h... yes 2024-04-25T17:12:27.568 INFO:tasks.workunit.client.0.smithi096.stdout:checking dirent.h usability... yes 2024-04-25T17:12:27.585 INFO:tasks.workunit.client.0.smithi096.stdout:checking dirent.h presence... yes 2024-04-25T17:12:27.585 INFO:tasks.workunit.client.0.smithi096.stdout:checking for dirent.h... yes 2024-04-25T17:12:27.635 INFO:tasks.workunit.client.0.smithi096.stdout:checking errno.h usability... yes 2024-04-25T17:12:27.652 INFO:tasks.workunit.client.0.smithi096.stdout:checking errno.h presence... yes 2024-04-25T17:12:27.652 INFO:tasks.workunit.client.0.smithi096.stdout:checking for errno.h... yes 2024-04-25T17:12:27.702 INFO:tasks.workunit.client.0.smithi096.stdout:checking malloc.h usability... yes 2024-04-25T17:12:27.722 INFO:tasks.workunit.client.0.smithi096.stdout:checking malloc.h presence... yes 2024-04-25T17:12:27.722 INFO:tasks.workunit.client.0.smithi096.stdout:checking for malloc.h... yes 2024-04-25T17:12:27.773 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid.h usability... no 2024-04-25T17:12:27.803 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid.h presence... no 2024-04-25T17:12:27.803 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid.h... no 2024-04-25T17:12:27.852 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid/uuid.h usability... yes 2024-04-25T17:12:27.871 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid/uuid.h presence... yes 2024-04-25T17:12:27.871 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid/uuid.h... yes 2024-04-25T17:12:27.921 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/uuid.h usability... no 2024-04-25T17:12:27.953 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/uuid.h presence... no 2024-04-25T17:12:27.953 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/uuid.h... no 2024-04-25T17:12:28.004 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/file.h usability... yes 2024-04-25T17:12:28.026 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/file.h presence... yes 2024-04-25T17:12:28.026 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/file.h... yes 2024-04-25T17:12:28.077 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fcntl.h usability... yes 2024-04-25T17:12:28.099 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fcntl.h presence... yes 2024-04-25T17:12:28.099 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/fcntl.h... yes 2024-04-25T17:12:28.155 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/syssgi.h usability... no 2024-04-25T17:12:28.185 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/syssgi.h presence... no 2024-04-25T17:12:28.186 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/syssgi.h... no 2024-04-25T17:12:28.253 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/param.h usability... yes 2024-04-25T17:12:28.274 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/param.h presence... yes 2024-04-25T17:12:28.274 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/param.h... yes 2024-04-25T17:12:28.289 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T17:12:28.325 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/statvfs.h usability... yes 2024-04-25T17:12:28.342 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/statvfs.h presence... yes 2024-04-25T17:12:28.342 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/statvfs.h... yes 2024-04-25T17:12:28.387 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/time.h usability... yes 2024-04-25T17:12:28.405 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/time.h presence... yes 2024-04-25T17:12:28.405 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/time.h... yes 2024-04-25T17:12:28.450 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/ioctl.h usability... yes 2024-04-25T17:12:28.468 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/ioctl.h presence... yes 2024-04-25T17:12:28.468 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/ioctl.h... yes 2024-04-25T17:12:28.525 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/wait.h usability... yes 2024-04-25T17:12:28.543 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/wait.h presence... yes 2024-04-25T17:12:28.543 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/wait.h... yes 2024-04-25T17:12:28.558 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/types.h... (cached) yes 2024-04-25T17:12:28.562 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strings.h... (cached) yes 2024-04-25T17:12:28.598 INFO:tasks.workunit.client.0.smithi096.stdout:checking err.h usability... yes 2024-04-25T17:12:28.616 INFO:tasks.workunit.client.0.smithi096.stdout:checking err.h presence... yes 2024-04-25T17:12:28.616 INFO:tasks.workunit.client.0.smithi096.stdout:checking for err.h... yes 2024-04-25T17:12:28.691 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/falloc.h usability... yes 2024-04-25T17:12:28.708 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/falloc.h presence... yes 2024-04-25T17:12:28.708 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/falloc.h... yes 2024-04-25T17:12:28.761 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-25T17:12:28.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:28 smithi171 ceph-mon[33189]: pgmap v69: 97 pgs: 97 active+clean; 40 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 892 op/s 2024-04-25T17:12:28.786 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:28 smithi097 ceph-mon[32602]: pgmap v69: 97 pgs: 97 active+clean; 40 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 892 op/s 2024-04-25T17:12:28.786 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[62520]: *** Caught signal (Segmentation fault) ** 2024-04-25T17:12:28.786 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[62520]: in thread 7ffbcf615700 thread_name:MR_Finisher 2024-04-25T17:12:28.786 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:28 smithi097 podman[66090]: 2024-04-25 17:12:28.675826288 +0000 UTC m=+0.020769490 container died 8ee9a110c54048c4924153b707e1f920be8be1d841cf402d0da9a2bd0778710d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T17:12:28.790 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-25T17:12:28.791 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T17:12:28.833 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-25T17:12:28.862 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-25T17:12:28.862 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T17:12:28.906 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/platform_defs.h usability... no 2024-04-25T17:12:28.935 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/platform_defs.h presence... no 2024-04-25T17:12:28.935 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/platform_defs.h... no 2024-04-25T17:12:28.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:28 smithi096 ceph-mon[29131]: pgmap v69: 97 pgs: 97 active+clean; 40 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.7 MiB/s wr, 892 op/s 2024-04-25T17:12:28.978 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfs/ioctl.h usability... no 2024-04-25T17:12:29.009 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfs/ioctl.h presence... no 2024-04-25T17:12:29.009 INFO:tasks.workunit.client.0.smithi096.stdout:checking for btrfs/ioctl.h... no 2024-04-25T17:12:29.053 INFO:tasks.workunit.client.0.smithi096.stdout:checking cifs/ioctl.h usability... no 2024-04-25T17:12:29.082 INFO:tasks.workunit.client.0.smithi096.stdout:checking cifs/ioctl.h presence... no 2024-04-25T17:12:29.083 INFO:tasks.workunit.client.0.smithi096.stdout:checking for cifs/ioctl.h... no 2024-04-25T17:12:29.127 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/mman.h usability... yes 2024-04-25T17:12:29.144 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/mman.h presence... yes 2024-04-25T17:12:29.144 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/mman.h... yes 2024-04-25T17:12:29.195 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/ext4.h usability... no 2024-04-25T17:12:29.223 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/ext4.h presence... no 2024-04-25T17:12:29.223 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/ext4.h... no 2024-04-25T17:12:29.251 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T17:12:29.306 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid_compare... no 2024-04-25T17:12:29.345 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T17:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 -- 172.21.15.96:0/4102368285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d0108a80 msgr2=0x7f46d0102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/4102368285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d0108a80 0x7f46d0102620 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009ca0 tx=0x7f46c802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 -- 172.21.15.96:0/4102368285 shutdown_connections 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/4102368285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46d01031d0 0x7f46d010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/4102368285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46d0108f20 0x7f46d0102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/4102368285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d0108a80 0x7f46d0102620 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 -- 172.21.15.96:0/4102368285 >> 172.21.15.96:0/4102368285 conn(0x7f46d0079170 msgr2=0x7f46d00795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 -- 172.21.15.96:0/4102368285 shutdown_connections 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.385+0000 7f46d7bb4640 1 -- 172.21.15.96:0/4102368285 wait complete. 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 Processor -- start 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 -- start start 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d01031d0 0x7f46d019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46d0108a80 0x7f46d019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46d0108f20 0x7f46d019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0112c70 con 0x7f46d01031d0 2024-04-25T17:12:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0112af0 con 0x7f46d0108f20 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d7bb4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0112df0 con 0x7f46d0108a80 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d01031d0 0x7f46d019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d01031d0 0x7f46d019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43416/0 (socket says 172.21.15.96:43416) 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 -- 172.21.15.96:0/1418253488 learned_addr learned my addr 172.21.15.96:0/1418253488 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d514f640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46d0108a80 0x7f46d019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d6151640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46d0108f20 0x7f46d019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 -- 172.21.15.96:0/1418253488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46d0108a80 msgr2=0x7f46d019ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46d0108a80 0x7f46d019ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 -- 172.21.15.96:0/1418253488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46d0108f20 msgr2=0x7f46d019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46d0108f20 0x7f46d019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.386+0000 7f46d5950640 1 -- 172.21.15.96:0/1418253488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d019f580 con 0x7f46d01031d0 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.387+0000 7f46d5950640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d01031d0 0x7f46d019a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097b0 tx=0x7f46c80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.387+0000 7f46c6ffd640 1 -- 172.21.15.96:0/1418253488 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8003bf0 con 0x7f46d01031d0 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.387+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d019f810 con 0x7f46d01031d0 2024-04-25T17:12:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.387+0000 7f46c6ffd640 1 -- 172.21.15.96:0/1418253488 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c8003d90 con 0x7f46d01031d0 2024-04-25T17:12:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.387+0000 7f46c6ffd640 1 -- 172.21.15.96:0/1418253488 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c802fc20 con 0x7f46d01031d0 2024-04-25T17:12:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.387+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d019fd60 con 0x7f46d01031d0 2024-04-25T17:12:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.389+0000 7f46c6ffd640 1 -- 172.21.15.96:0/1418253488 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f46c80305c0 con 0x7f46d01031d0 2024-04-25T17:12:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.389+0000 7f46dea61740 1 Processor -- start 2024-04-25T17:12:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46dea61740 1 -- start start 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46dea61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77782980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46dea61740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77782ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46dea61740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773c24d0 0x557f77783400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46dea61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557f776e9460 con 0x557f773e31b0 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46dea61740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f776a3590 con 0x557f773c24d0 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46dea61740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557f776e5420 con 0x557f773ba2f0 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d6151640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77782980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d6151640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77782980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43422/0 (socket says 172.21.15.96:43422) 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d6151640 1 -- 172.21.15.96:0/201380427 learned_addr learned my addr 172.21.15.96:0/201380427 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d514f640 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773c24d0 0x557f77783400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d5950640 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77782ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46c6ffd640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f46bc0afc50 0x7f46bc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d6151640 1 -- 172.21.15.96:0/201380427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 msgr2=0x557f77782ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d6151640 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77782ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d6151640 1 -- 172.21.15.96:0/201380427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773c24d0 msgr2=0x557f77783400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.390+0000 7f46d6151640 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773c24d0 0x557f77783400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.391+0000 7f46d6151640 1 -- 172.21.15.96:0/201380427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f776b00d0 con 0x557f773e31b0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.391+0000 7f46d6151640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f46bc0afc50 0x7f46bc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.391+0000 7f46c6ffd640 1 -- 172.21.15.96:0/1418253488 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f46c8100530 con 0x7f46d01031d0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.391+0000 7f46d6151640 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77782980 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00ddb0 tx=0x7f46cc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a0f48b93acd94b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.391+0000 7f46bbfff640 1 -- 172.21.15.96:0/201380427 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc01c600 con 0x557f773e31b0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.391+0000 7f46bbfff640 1 -- 172.21.15.96:0/201380427 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc01cc50 con 0x557f773e31b0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.391+0000 7f46bbfff640 1 -- 172.21.15.96:0/201380427 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc01bb40 con 0x557f773e31b0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 -- 172.21.15.96:0/201380427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 msgr2=0x557f77782980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46d6151640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f46bc0afc50 0x7f46bc0b2160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46cc020000 tx=0x7f46cc01d000 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77782980 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00ddb0 tx=0x7f46cc00f730 comp rx=0 tx=0).stop 2024-04-25T17:12:29.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 -- 172.21.15.96:0/201380427 shutdown_connections 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773c24d0 0x557f77783400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77782ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 --2- 172.21.15.96:0/201380427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77782980 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 -- 172.21.15.96:0/201380427 >> 172.21.15.96:0/201380427 conn(0x557f773d7300 msgr2=0x557f7777f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 -- 172.21.15.96:0/201380427 shutdown_connections 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 -- 172.21.15.96:0/201380427 wait complete. 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 -- start start 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.392+0000 7f46dea61740 1 Processor -- start 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46dea61740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773ba2f0 0x557f777915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46dea61740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773c24d0 0x557f77791b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46dea61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77797580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46dea61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557f77794090 con 0x557f773e31b0 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46dea61740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f77670b60 con 0x557f773ba2f0 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46dea61740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557f77794210 con 0x557f773c24d0 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46dea61740 10 client.0 _ll_register_callbacks cb 0x557f776952b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77797580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d5950640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773ba2f0 0x557f777915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77797580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43432/0 (socket says 172.21.15.96:43432) 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 -- 172.21.15.96:0/3589525470 learned_addr learned my addr 172.21.15.96:0/3589525470 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d514f640 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773c24d0 0x557f77791b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 -- 172.21.15.96:0/3589525470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773c24d0 msgr2=0x557f77791b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773c24d0 0x557f77791b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 -- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773ba2f0 msgr2=0x557f777915d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773ba2f0 0x557f777915d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 -- 172.21.15.96:0/3589525470 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f773fb210 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d5950640 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773ba2f0 0x557f777915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.393+0000 7f46d6151640 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77797580 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00dd80 tx=0x7f46cc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42eb24e12bd91747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46b97fa640 1 -- 172.21.15.96:0/3589525470 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc034060 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46b97fa640 1 -- 172.21.15.96:0/3589525470 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f46cc022040 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46cc01cc50 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46b97fa640 1 -- 172.21.15.96:0/3589525470 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f46cc004600 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 10 client.15525 fetch_fsmap learned FSMap version 18 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x557f774c7620 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46b97fa640 1 -- 172.21.15.96:0/3589525470 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc01cc50 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46b97fa640 1 -- 172.21.15.96:0/3589525470 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f46cc026480 con 0x557f773e31b0 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 10 client.15525 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 20 client.15525 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773a1d80 0x557f777a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 4 client.15525 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.394+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f776f7070 con 0x557f773a1d80 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.395+0000 7f46d5950640 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773a1d80 0x557f777a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.395+0000 7f46d5950640 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773a1d80 0x557f777a3ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f3f0561680261d server_cookie=5c6d28e0937a5972 in_seq=0 out_seq=0 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.395+0000 7f46b97fa640 10 client.15525 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46b97fa640 1 -- 172.21.15.96:0/3589525470 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f776f7070 con 0x557f773a1d80 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46b97fa640 10 client.15525 handle_command_reply: tid=0 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 client.15525 shutdown 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 msgr2=0x557f77797580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77797580 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00dd80 tx=0x7f46cc00d4a0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 shutdown_connections 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773a1d80 0x557f777a3ac0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773e31b0 0x557f77797580 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773c24d0 0x557f77791b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 --2- 172.21.15.96:0/3589525470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773ba2f0 0x557f777915d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46b97fa640 0 client.15525 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 >> 172.21.15.96:0/3589525470 conn(0x557f773d7300 msgr2=0x557f77783330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 shutdown_connections 2024-04-25T17:12:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 1 -- 172.21.15.96:0/3589525470 wait complete. 2024-04-25T17:12:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.396+0000 7f46dea61740 20 client.15525 trim_cache size 0 max 16384 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 Processor -- start 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 -- start start 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77797440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f7778ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f7778b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01ab790 con 0x557f773c24d0 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01adff0 con 0x557f773e31b0 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.411+0000 7f46dea61740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01abde0 con 0x557f773ba2f0 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f7778ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f7778ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43438/0 (socket says 172.21.15.96:43438) 2024-04-25T17:12:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 -- 172.21.15.96:0/2618564708 learned_addr learned my addr 172.21.15.96:0/2618564708 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d6151640 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f7778b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d5950640 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77797440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 -- 172.21.15.96:0/2618564708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 msgr2=0x557f77797440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77797440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 -- 172.21.15.96:0/2618564708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 msgr2=0x557f7778b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f7778b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 -- 172.21.15.96:0/2618564708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f777940e0 con 0x557f773c24d0 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d6151640 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f7778b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46d514f640 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f7778ad30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f46d006c4b0 tx=0x7f46c0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a715275022ba6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46bb7fe640 1 -- 172.21.15.96:0/2618564708 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0011d80 con 0x557f773c24d0 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46bb7fe640 1 -- 172.21.15.96:0/2618564708 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c0005020 con 0x557f773c24d0 2024-04-25T17:12:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.412+0000 7f46bb7fe640 1 -- 172.21.15.96:0/2618564708 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0004ad0 con 0x557f773c24d0 2024-04-25T17:12:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 -- 172.21.15.96:0/2618564708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 msgr2=0x557f7778ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f7778ad30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f46d006c4b0 tx=0x7f46c0011440 comp rx=0 tx=0).stop 2024-04-25T17:12:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 -- 172.21.15.96:0/2618564708 shutdown_connections 2024-04-25T17:12:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f7778b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f7778ad30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 --2- 172.21.15.96:0/2618564708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77797440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 -- 172.21.15.96:0/2618564708 >> 172.21.15.96:0/2618564708 conn(0x557f773d7300 msgr2=0x557f7778fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 -- 172.21.15.96:0/2618564708 shutdown_connections 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.413+0000 7f46dea61740 1 -- 172.21.15.96:0/2618564708 wait complete. 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 -- start start 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 Processor -- start 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77798350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f77798890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f77798dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01ac4b0 con 0x557f773c24d0 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01ad920 con 0x557f773e31b0 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01acb80 con 0x557f773ba2f0 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46dea61740 10 client.0 _ll_register_callbacks cb 0x7f46cc01f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46d514f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f77798890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46d514f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f77798890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43442/0 (socket says 172.21.15.96:43442) 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.414+0000 7f46d514f640 1 -- 172.21.15.96:0/3409443504 learned_addr learned my addr 172.21.15.96:0/3409443504 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d6151640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f77798dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d5950640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77798350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d514f640 1 -- 172.21.15.96:0/3409443504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 msgr2=0x557f77798350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d514f640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77798350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d514f640 1 -- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 msgr2=0x557f77798dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d514f640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f77798dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d514f640 1 -- 172.21.15.96:0/3409443504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f7779d5a0 con 0x557f773c24d0 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d5950640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77798350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46d514f640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f77798890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f46c0011940 tx=0x7f46c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19ec32475c28760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46b97fa640 1 -- 172.21.15.96:0/3409443504 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0027020 con 0x557f773c24d0 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46b97fa640 1 -- 172.21.15.96:0/3409443504 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8498+0+0 (secure 0 0 0) 0x7f46c0004df0 con 0x557f773c24d0 2024-04-25T17:12:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c0005020 con 0x557f773c24d0 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46b97fa640 1 -- 172.21.15.96:0/3409443504 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f46c0032050 con 0x557f773c24d0 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46dea61740 10 client.15531 fetch_fsmap learned FSMap version 18 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.415+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=4+}) -- 0x557f7779e810 con 0x557f773c24d0 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46b97fa640 1 -- 172.21.15.96:0/3409443504 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46c0005020 con 0x557f773c24d0 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46b97fa640 1 -- 172.21.15.96:0/3409443504 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 18) ==== 8322+0+0 (secure 0 0 0) 0x7f46c0023520 con 0x557f773c24d0 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46dea61740 10 client.15531 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46dea61740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46dea61740 20 client.15531 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46dea61740 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773771c0 0x557f7780dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46dea61740 4 client.15531 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f776f0ee0 con 0x557f773771c0 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.416+0000 7f46d5950640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773771c0 0x557f7780dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.417+0000 7f46b97fa640 10 client.15531 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.417+0000 7f46d5950640 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773771c0 0x557f7780dfc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97cf4c89f6192527 server_cookie=d01babeed0fe660b in_seq=0 out_seq=0 2024-04-25T17:12:29.421 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/xfs.h... yes 2024-04-25T17:12:29.450 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/libxfs.h... no 2024-04-25T17:12:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46b97fa640 1 -- 172.21.15.96:0/3409443504 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f776f0ee0 con 0x557f773771c0 2024-04-25T17:12:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46b97fa640 10 client.15531 handle_command_reply: tid=0 2024-04-25T17:12:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 client.15531 shutdown 2024-04-25T17:12:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 msgr2=0x557f77798890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f77798890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f46c0011940 tx=0x7f46c0004a40 comp rx=0 tx=0).stop 2024-04-25T17:12:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 shutdown_connections 2024-04-25T17:12:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f773771c0 0x557f7780dfc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f773e31b0 0x557f77798dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46b97fa640 0 client.15531 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f773c24d0 0x557f77798890 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 --2- 172.21.15.96:0/3409443504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f773ba2f0 0x557f77798350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.905+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 >> 172.21.15.96:0/3409443504 conn(0x557f773d7300 msgr2=0x557f77782880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 shutdown_connections 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46dea61740 1 -- 172.21.15.96:0/3409443504 wait complete. 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46dea61740 20 client.15531 trim_cache size 0 max 16384 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:12:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:12:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f46bc0afc50 msgr2=0x7f46bc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f46bc0afc50 0x7f46bc0b2160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46cc020000 tx=0x7f46cc01d000 comp rx=0 tx=0).stop 2024-04-25T17:12:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d01031d0 msgr2=0x7f46d019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d01031d0 0x7f46d019a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097b0 tx=0x7f46c80319f0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 shutdown_connections 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f46bc0afc50 0x7f46bc0b2160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46d0108f20 0x7f46d019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f46d0108a80 0x7f46d019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 --2- 172.21.15.96:0/1418253488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d01031d0 0x7f46d019a7a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.906+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 >> 172.21.15.96:0/1418253488 conn(0x7f46d0079170 msgr2=0x7f46d0074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.907+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 shutdown_connections 2024-04-25T17:12:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:29.907+0000 7f46d7bb4640 1 -- 172.21.15.96:0/1418253488 wait complete. 2024-04-25T17:12:29.910 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T17:12:29.948 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/xqm.h... yes 2024-04-25T17:12:29.986 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:29 smithi097 podman[66090]: 2024-04-25 17:12:29.671508823 +0000 UTC m=+1.016452034 container remove 8ee9a110c54048c4924153b707e1f920be8be1d841cf402d0da9a2bd0778710d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T17:12:29.986 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:29 smithi097 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=139/n/a 2024-04-25T17:12:29.999 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T17:12:30.036 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/handle.h usability... yes 2024-04-25T17:12:30.052 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/handle.h presence... yes 2024-04-25T17:12:30.052 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/handle.h... yes 2024-04-25T17:12:30.091 INFO:tasks.workunit.client.0.smithi096.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T17:12:30.137 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/acl.h usability... yes 2024-04-25T17:12:30.155 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/acl.h presence... yes 2024-04-25T17:12:30.155 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/acl.h... yes 2024-04-25T17:12:30.186 INFO:tasks.workunit.client.0.smithi096.stdout:checking acl/libacl.h usability... yes 2024-04-25T17:12:30.203 INFO:tasks.workunit.client.0.smithi096.stdout:checking acl/libacl.h presence... yes 2024-04-25T17:12:30.204 INFO:tasks.workunit.client.0.smithi096.stdout:checking for acl/libacl.h... yes 2024-04-25T17:12:30.244 INFO:tasks.workunit.client.0.smithi096.stdout:checking for acl_init in -lacl... yes 2024-04-25T17:12:30.260 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:30 smithi097 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-25T17:12:30.260 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:30 smithi097 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.005s CPU time. 2024-04-25T17:12:30.300 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm-ndbm.h usability... no 2024-04-25T17:12:30.335 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm-ndbm.h presence... no 2024-04-25T17:12:30.335 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gdbm-ndbm.h... no 2024-04-25T17:12:30.376 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm/ndbm.h usability... no 2024-04-25T17:12:30.405 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm/ndbm.h presence... no 2024-04-25T17:12:30.405 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gdbm/ndbm.h... no 2024-04-25T17:12:30.444 INFO:tasks.workunit.client.0.smithi096.stdout:checking ndbm.h usability... no 2024-04-25T17:12:30.483 INFO:tasks.workunit.client.0.smithi096.stdout:checking ndbm.h presence... no 2024-04-25T17:12:30.483 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ndbm.h... no 2024-04-25T17:12:30.518 INFO:tasks.workunit.client.0.smithi096.stdout:checking libaio.h usability... yes 2024-04-25T17:12:30.543 INFO:tasks.workunit.client.0.smithi096.stdout:checking libaio.h presence... yes 2024-04-25T17:12:30.543 INFO:tasks.workunit.client.0.smithi096.stdout:checking for libaio.h... yes 2024-04-25T17:12:30.595 INFO:tasks.workunit.client.0.smithi096.stdout:checking liburing.h usability... no 2024-04-25T17:12:30.631 INFO:tasks.workunit.client.0.smithi096.stdout:checking liburing.h presence... no 2024-04-25T17:12:30.631 INFO:tasks.workunit.client.0.smithi096.stdout:checking for liburing.h... no 2024-04-25T17:12:30.675 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/capability.h usability... no 2024-04-25T17:12:30.711 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/capability.h presence... no 2024-04-25T17:12:30.711 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/capability.h... no 2024-04-25T17:12:30.747 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fiemap.h usability... yes 2024-04-25T17:12:30.768 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fiemap.h presence... yes 2024-04-25T17:12:30.768 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/fiemap.h... yes 2024-04-25T17:12:30.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:30 smithi171 ceph-mon[33189]: pgmap v70: 97 pgs: 97 active+clean; 40 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 872 op/s 2024-04-25T17:12:30.805 INFO:tasks.workunit.client.0.smithi096.stdout:checking for fallocate... yes 2024-04-25T17:12:30.861 INFO:tasks.workunit.client.0.smithi096.stdout:checking for open_by_handle_at... yes 2024-04-25T17:12:30.913 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fs.h usability... yes 2024-04-25T17:12:30.935 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fs.h presence... yes 2024-04-25T17:12:30.935 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/fs.h... yes 2024-04-25T17:12:30.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:30 smithi096 ceph-mon[29131]: pgmap v70: 97 pgs: 97 active+clean; 40 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 872 op/s 2024-04-25T17:12:30.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:30 smithi097 ceph-mon[32602]: pgmap v70: 97 pgs: 97 active+clean; 40 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 872 op/s 2024-04-25T17:12:30.998 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfsutil.h usability... no 2024-04-25T17:12:31.033 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfsutil.h presence... no 2024-04-25T17:12:31.033 INFO:tasks.workunit.client.0.smithi096.stdout:checking for btrfsutil.h... no 2024-04-25T17:12:31.080 INFO:tasks.workunit.client.0.smithi096.stdout:checking for copy_file_range... yes 2024-04-25T17:12:31.128 INFO:tasks.workunit.client.0.smithi096.stdout:checking for SEEK_DATA... yes 2024-04-25T17:12:31.189 INFO:tasks.workunit.client.0.smithi096.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T17:12:31.238 INFO:tasks.workunit.client.0.smithi096.stdout:checking for nftw... yes 2024-04-25T17:12:31.292 INFO:tasks.workunit.client.0.smithi096.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T17:12:31.341 INFO:tasks.workunit.client.0.smithi096.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T17:12:31.407 INFO:tasks.workunit.client.0.smithi096.stdout:checking for FICLONE... yes 2024-04-25T17:12:31.454 INFO:tasks.workunit.client.0.smithi096.stdout:checking for renameat2... yes 2024-04-25T17:12:31.495 INFO:tasks.workunit.client.0.smithi096.stdout:checking for reallocarray... yes 2024-04-25T17:12:31.561 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct mount_attr... yes 2024-04-25T17:12:31.634 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T17:12:31.700 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T17:12:31.765 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T17:12:31.824 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T17:12:31.881 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T17:12:31.939 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T17:12:31.996 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T17:12:32.035 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs.h usability... yes 2024-04-25T17:12:32.052 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs.h presence... yes 2024-04-25T17:12:32.052 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/btrfs.h... yes 2024-04-25T17:12:32.100 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-25T17:12:32.117 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-25T17:12:32.117 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T17:12:32.156 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T17:12:32.186 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T17:12:32.206 INFO:tasks.workunit.client.0.smithi096.stdout:configure: creating ./config.status 2024-04-25T17:12:32.533 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating include/builddefs 2024-04-25T17:12:32.546 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating include/config.h 2024-04-25T17:12:32.556 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: executing libtool commands 2024-04-25T17:12:32.629 INFO:tasks.workunit.client.0.smithi096.stdout:Building include 2024-04-25T17:12:32.630 INFO:tasks.workunit.client.0.smithi096.stdout:Building lib 2024-04-25T17:12:32.630 INFO:tasks.workunit.client.0.smithi096.stdout:Building tests 2024-04-25T17:12:32.634 INFO:tasks.workunit.client.0.smithi096.stdout: [LTDEP] 2024-04-25T17:12:32.636 INFO:tasks.workunit.client.0.smithi096.stdout:Building btrfs 2024-04-25T17:12:32.637 INFO:tasks.workunit.client.0.smithi096.stdout:Building ceph 2024-04-25T17:12:32.637 INFO:tasks.workunit.client.0.smithi096.stdout:Building cifs 2024-04-25T17:12:32.646 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-25T17:12:32.647 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-25T17:12:32.660 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-25T17:12:32.662 INFO:tasks.workunit.client.0.smithi096.stdout:Building ext4 2024-04-25T17:12:32.664 INFO:tasks.workunit.client.0.smithi096.stdout:Building f2fs 2024-04-25T17:12:32.672 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-25T17:12:32.679 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-25T17:12:32.692 INFO:tasks.workunit.client.0.smithi096.stdout:Building generic 2024-04-25T17:12:32.720 INFO:tasks.workunit.client.0.smithi096.stdout:Building nfs 2024-04-25T17:12:32.726 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-25T17:12:32.728 INFO:tasks.workunit.client.0.smithi096.stdout:Building ocfs2 2024-04-25T17:12:32.734 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-25T17:12:32.744 INFO:tasks.workunit.client.0.smithi096.stdout:Building overlay 2024-04-25T17:12:32.751 INFO:tasks.workunit.client.0.smithi096.stdout:Building perf 2024-04-25T17:12:32.757 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-25T17:12:32.769 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-25T17:12:32.770 INFO:tasks.workunit.client.0.smithi096.stdout:Building selftest 2024-04-25T17:12:32.777 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-25T17:12:32.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:32 smithi171 ceph-mon[33189]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 1.21k op/s 2024-04-25T17:12:32.791 INFO:tasks.workunit.client.0.smithi096.stdout:Building shared 2024-04-25T17:12:32.792 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-25T17:12:32.798 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-25T17:12:32.814 INFO:tasks.workunit.client.0.smithi096.stdout:Building tmpfs 2024-04-25T17:12:32.819 INFO:tasks.workunit.client.0.smithi096.stdout:Building udf 2024-04-25T17:12:32.821 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-25T17:12:32.822 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dataascii.lo 2024-04-25T17:12:32.825 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-25T17:12:32.837 INFO:tasks.workunit.client.0.smithi096.stdout:Building xfs 2024-04-25T17:12:32.839 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] databin.lo 2024-04-25T17:12:32.908 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] datapid.lo 2024-04-25T17:12:32.922 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-25T17:12:32.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:32 smithi096 ceph-mon[29131]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 1.21k op/s 2024-04-25T17:12:32.950 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] file_lock.lo 2024-04-25T17:12:32.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:32 smithi097 ceph-mon[32602]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 1.21k op/s 2024-04-25T17:12:32.993 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] forker.lo 2024-04-25T17:12:33.024 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] pattern.lo 2024-04-25T17:12:33.033 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] open_flags.lo 2024-04-25T17:12:33.105 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] random_range.lo 2024-04-25T17:12:33.146 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] string_to_tokens.lo 2024-04-25T17:12:33.149 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] str_to_bytes.lo 2024-04-25T17:12:33.196 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] tlibio.lo 2024-04-25T17:12:33.257 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] write_log.lo 2024-04-25T17:12:33.308 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] random.lo 2024-04-25T17:12:33.504 INFO:tasks.workunit.client.0.smithi096.stdout: [LD] 2024-04-25T17:12:33.628 INFO:tasks.workunit.client.0.smithi096.stdout:Building ltp 2024-04-25T17:12:33.628 INFO:tasks.workunit.client.0.smithi096.stdout:Building src 2024-04-25T17:12:33.628 INFO:tasks.workunit.client.0.smithi096.stdout:Building m4 2024-04-25T17:12:33.628 INFO:tasks.workunit.client.0.smithi096.stdout:Building common 2024-04-25T17:12:33.633 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T17:12:33.647 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T17:12:33.813 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] doio 2024-04-25T17:12:33.813 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsstress 2024-04-25T17:12:33.814 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsx 2024-04-25T17:12:34.466 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] growfiles 2024-04-25T17:12:34.565 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] iogen 2024-04-25T17:12:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:34 smithi096 ceph-mon[29131]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-25T17:12:34.940 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-stress 2024-04-25T17:12:34.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:34 smithi097 ceph-mon[32602]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-25T17:12:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:34 smithi171 ceph-mon[33189]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-25T17:12:35.503 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T17:12:35.503 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T17:12:35.503 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T17:12:35.504 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T17:12:35.725 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dirstress 2024-04-25T17:12:35.725 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fill 2024-04-25T17:12:35.725 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fill2 2024-04-25T17:12:35.725 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] getpagesize 2024-04-25T17:12:35.851 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] holes 2024-04-25T17:12:35.880 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] lstat64 2024-04-25T17:12:35.955 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] nametest 2024-04-25T17:12:35.964 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] permname 2024-04-25T17:12:36.048 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] randholes 2024-04-25T17:12:36.064 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] runas 2024-04-25T17:12:36.183 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] truncfile 2024-04-25T17:12:36.213 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] usemem 2024-04-25T17:12:36.252 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mmapcat 2024-04-25T17:12:36.323 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] append_reader 2024-04-25T17:12:36.358 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] append_writer 2024-04-25T17:12:36.370 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dirperf 2024-04-25T17:12:36.389 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] metaperf 2024-04-25T17:12:36.472 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] devzero 2024-04-25T17:12:36.500 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] feature 2024-04-25T17:12:36.605 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] alloc 2024-04-25T17:12:36.655 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fault 2024-04-25T17:12:36.672 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fstest 2024-04-25T17:12:36.728 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_access_root 2024-04-25T17:12:36.816 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] godown 2024-04-25T17:12:36.818 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] resvtest 2024-04-25T17:12:36.882 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] writemod 2024-04-25T17:12:36.919 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] writev_on_pagefault 2024-04-25T17:12:36.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:36 smithi096 ceph-mon[29131]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-25T17:12:36.982 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] makeextents 2024-04-25T17:12:36.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:36 smithi097 ceph-mon[32602]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-25T17:12:36.998 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] itrash 2024-04-25T17:12:37.023 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] rename 2024-04-25T17:12:37.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:36 smithi171 ceph-mon[33189]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.1 MiB/s wr, 343 op/s 2024-04-25T17:12:37.074 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] multi_open_unlink 2024-04-25T17:12:37.141 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] unwritten_sync 2024-04-25T17:12:37.154 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] genhashnames 2024-04-25T17:12:37.158 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_holes 2024-04-25T17:12:37.242 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_writev 2024-04-25T17:12:37.300 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_truncate_cmtime 2024-04-25T17:12:37.319 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dirhash_collide 2024-04-25T17:12:37.365 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_rename_overwrite 2024-04-25T17:12:37.400 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] holetest 2024-04-25T17:12:37.450 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_truncate_self 2024-04-25T17:12:37.503 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] af_unix 2024-04-25T17:12:37.521 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_stale_pmd 2024-04-25T17:12:37.579 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_cow_race 2024-04-25T17:12:37.631 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_fallocate 2024-04-25T17:12:37.645 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsync-err 2024-04-25T17:12:37.669 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_write_ro 2024-04-25T17:12:37.739 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T17:12:37.785 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T17:12:37.817 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_ofd_locks 2024-04-25T17:12:37.831 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_collision 2024-04-25T17:12:37.898 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mmap-write-concurrent 2024-04-25T17:12:37.943 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_get_file_time 2024-04-25T17:12:38.003 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_create_short_dirs 2024-04-25T17:12:38.046 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_create_long_dirs 2024-04-25T17:12:38.056 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_enospc 2024-04-25T17:12:38.094 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_writev_overlap 2024-04-25T17:12:38.172 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] checkpoint_journal 2024-04-25T17:12:38.212 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mmap-rw-fault 2024-04-25T17:12:38.246 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] allocstale 2024-04-25T17:12:38.270 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T17:12:38.344 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fake-dump-rootino 2024-04-25T17:12:38.405 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dio-buf-fault 2024-04-25T17:12:38.422 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] rewinddir-test 2024-04-25T17:12:38.437 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] readdir-while-renames 2024-04-25T17:12:38.500 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] xfsctl 2024-04-25T17:12:38.557 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bstat 2024-04-25T17:12:38.586 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mtab 2024-04-25T17:12:38.608 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] getdevicesize 2024-04-25T17:12:38.696 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T17:12:38.741 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T17:12:38.765 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] ftrunc 2024-04-25T17:12:38.776 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] trunc 2024-04-25T17:12:38.853 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fs_perms 2024-04-25T17:12:38.892 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] testx 2024-04-25T17:12:38.903 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] looptest 2024-04-25T17:12:38.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:38 smithi096 ceph-mon[29131]: pgmap v74: 97 pgs: 97 active+clean; 28 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.2 MiB/s wr, 355 op/s 2024-04-25T17:12:38.944 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] locktest 2024-04-25T17:12:38.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:38 smithi097 ceph-mon[32602]: pgmap v74: 97 pgs: 97 active+clean; 28 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.2 MiB/s wr, 355 op/s 2024-04-25T17:12:39.012 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] unwritten_mmap 2024-04-25T17:12:39.021 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bulkstat_unlink_test 2024-04-25T17:12:39.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:38 smithi171 ceph-mon[33189]: pgmap v74: 97 pgs: 97 active+clean; 28 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 1.2 MiB/s wr, 355 op/s 2024-04-25T17:12:39.104 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] deduperace 2024-04-25T17:12:39.180 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T17:12:39.230 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_dir_offset 2024-04-25T17:12:39.277 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_futimens 2024-04-25T17:12:39.313 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_immutable 2024-04-25T17:12:39.370 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] stale_handle 2024-04-25T17:12:39.386 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] pwrite_mmap_blocked 2024-04-25T17:12:39.402 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_dir_offset2 2024-04-25T17:12:39.526 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] seek_sanity_test 2024-04-25T17:12:39.558 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] seek_copy_test 2024-04-25T17:12:39.586 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_readdir_1 2024-04-25T17:12:39.731 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_readdir_2 2024-04-25T17:12:39.741 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsync-tester 2024-04-25T17:12:39.882 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] nsexec 2024-04-25T17:12:39.985 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] cloner 2024-04-25T17:12:39.996 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] renameat2 2024-04-25T17:12:40.028 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_getcwd 2024-04-25T17:12:40.043 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] e4compact 2024-04-25T17:12:40.161 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] test-nextquota 2024-04-25T17:12:40.162 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] punch-alternating 2024-04-25T17:12:40.196 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T17:12:40.261 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] listxattr 2024-04-25T17:12:40.330 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dio-interleaved 2024-04-25T17:12:40.335 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_dir_type 2024-04-25T17:12:40.363 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dio-invalidate-cache 2024-04-25T17:12:40.406 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] stat_test 2024-04-25T17:12:40.485 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T17:12:40.485 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] attr_replace_test 2024-04-25T17:12:40.505 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-04-25T17:12:40.505 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 systemd[1]: Stopped Ceph mds.h for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:12:40.505 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.005s CPU time. 2024-04-25T17:12:40.505 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 systemd[1]: Starting Ceph mds.h for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:12:40.505 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 podman[66193]: 2024-04-25 17:12:40.365462192 +0000 UTC m=+0.018192571 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:12:40.505 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 podman[66193]: 2024-04-25 17:12:40.505840016 +0000 UTC m=+0.158570365 container create 900c48e9406c92294cfe307f6f4beca06eba7fc111a442c53c70fdeb0e2bce29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T17:12:40.577 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] swapon 2024-04-25T17:12:40.652 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mkswap 2024-04-25T17:12:40.655 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_attr_corruption 2024-04-25T17:12:40.723 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_open_tmpfiles 2024-04-25T17:12:40.740 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fscrypt-crypt-util 2024-04-25T17:12:40.797 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bulkstat_null_ocount 2024-04-25T17:12:40.804 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] splice-test 2024-04-25T17:12:40.855 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:40 smithi097 ceph-mon[32602]: pgmap v75: 97 pgs: 97 active+clean; 28 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 620 KiB/s wr, 345 op/s 2024-04-25T17:12:40.907 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] chprojid_fail 2024-04-25T17:12:40.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:40 smithi096 ceph-mon[29131]: pgmap v75: 97 pgs: 97 active+clean; 28 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 620 KiB/s wr, 345 op/s 2024-04-25T17:12:40.965 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] detached_mounts_propagation 2024-04-25T17:12:40.981 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] ext4_resize 2024-04-25T17:12:41.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:40 smithi171 ceph-mon[33189]: pgmap v75: 97 pgs: 97 active+clean; 28 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 620 KiB/s wr, 345 op/s 2024-04-25T17:12:41.073 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_readdir_3 2024-04-25T17:12:41.130 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] splice2pipe 2024-04-25T17:12:41.139 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] uuid_ioctl 2024-04-25T17:12:41.234 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 podman[66193]: 2024-04-25 17:12:40.856024479 +0000 UTC m=+0.508754829 container init 900c48e9406c92294cfe307f6f4beca06eba7fc111a442c53c70fdeb0e2bce29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T17:12:41.234 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 podman[66193]: 2024-04-25 17:12:40.858805296 +0000 UTC m=+0.511535645 container start 900c48e9406c92294cfe307f6f4beca06eba7fc111a442c53c70fdeb0e2bce29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T17:12:41.234 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 ceph-mds[66212]: -- 172.21.15.97:0/3360275967 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a27b465a40 con 0x55a27b4ad180 2024-04-25T17:12:41.234 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[66208]: starting mds.h at 2024-04-25T17:12:41.234 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 bash[66193]: 900c48e9406c92294cfe307f6f4beca06eba7fc111a442c53c70fdeb0e2bce29 2024-04-25T17:12:41.234 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:12:40 smithi097 systemd[1]: Started Ceph mds.h for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:12:41.277 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T17:12:41.277 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fiemap-fault 2024-04-25T17:12:41.287 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fiemap-tester 2024-04-25T17:12:41.436 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_stripealign 2024-04-25T17:12:41.443 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] open_by_handle 2024-04-25T17:12:41.466 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_dio 2024-04-25T17:12:41.580 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] xfsfind 2024-04-25T17:12:41.589 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_reflink_read_race 2024-04-25T17:12:41.642 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fssum 2024-04-25T17:12:41.693 INFO:tasks.workunit.client.0.smithi096.stdout:Building log-writes 2024-04-25T17:12:41.698 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] replay-log 2024-04-25T17:12:41.733 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:41 smithi097 ceph-mon[32602]: daemon mds.h restarted 2024-04-25T17:12:41.781 INFO:tasks.workunit.client.0.smithi096.stdout:Building perf 2024-04-25T17:12:41.782 INFO:tasks.workunit.client.0.smithi096.stdout:Building aio-dio-regress 2024-04-25T17:12:41.785 INFO:tasks.workunit.client.0.smithi096.stdout:Building vfs 2024-04-25T17:12:41.788 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aiocp 2024-04-25T17:12:41.790 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] vfstest 2024-04-25T17:12:41.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:41 smithi096 ceph-mon[29131]: daemon mds.h restarted 2024-04-25T17:12:42.002 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T17:12:42.012 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mount-idmapped 2024-04-25T17:12:42.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:41 smithi171 ceph-mon[33189]: daemon mds.h restarted 2024-04-25T17:12:42.118 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T17:12:42.173 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-cow-race 2024-04-25T17:12:42.290 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-cycle-write 2024-04-25T17:12:42.323 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-eof-race 2024-04-25T17:12:42.406 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-extend-stat 2024-04-25T17:12:42.474 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-fcntl-race 2024-04-25T17:12:42.496 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T17:12:42.568 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T17:12:42.651 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T17:12:42.652 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aiodio_sparse2 2024-04-25T17:12:42.723 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T17:12:42.810 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-write-verify 2024-04-25T17:12:42.870 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T17:12:42.871 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T17:12:42.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:42 smithi096 ceph-mon[29131]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 645 KiB/s wr, 347 op/s 2024-04-25T17:12:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:42 smithi096 ceph-mon[29131]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T17:12:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:42 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6836/3526163799,v1:172.21.15.97:6837/3526163799] up:boot 2024-04-25T17:12:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:42 smithi096 ceph-mon[29131]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:12:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:42 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:12:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:42 smithi096 ceph-mon[29131]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:12:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:42 smithi097 ceph-mon[32602]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 645 KiB/s wr, 347 op/s 2024-04-25T17:12:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:42 smithi097 ceph-mon[32602]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T17:12:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:42 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6836/3526163799,v1:172.21.15.97:6837/3526163799] up:boot 2024-04-25T17:12:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:42 smithi097 ceph-mon[32602]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:12:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:42 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:12:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:42 smithi097 ceph-mon[32602]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:12:43.011 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 -- 172.21.15.96:0/649023685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410e0c0 msgr2=0x7f8c741105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/649023685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410e0c0 0x7f8c741105b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009ca0 tx=0x7f8c6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 -- 172.21.15.96:0/649023685 shutdown_connections 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/649023685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410e0c0 0x7f8c741105b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/649023685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c7410b6d0 0x7f8c7410daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/649023685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c74104c70 0x7f8c74105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 -- 172.21.15.96:0/649023685 >> 172.21.15.96:0/649023685 conn(0x7f8c740781b0 msgr2=0x7f8c740785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.025+0000 7f8c7b33f640 1 -- 172.21.15.96:0/649023685 shutdown_connections 2024-04-25T17:12:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 -- 172.21.15.96:0/649023685 wait complete. 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 Processor -- start 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 -- start start 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c74104c70 0x7f8c7419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410b6d0 0x7f8c7419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c7410e0c0 0x7f8c7419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74113d20 con 0x7f8c7410b6d0 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74113ba0 con 0x7f8c74104c70 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.026+0000 7f8c7b33f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74113ea0 con 0x7f8c7410e0c0 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410b6d0 0x7f8c7419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410b6d0 0x7f8c7419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35198/0 (socket says 172.21.15.96:35198) 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 -- 172.21.15.96:0/2898709480 learned_addr learned my addr 172.21.15.96:0/2898709480 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c790db640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c74104c70 0x7f8c7419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c798dc640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c7410e0c0 0x7f8c7419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 -- 172.21.15.96:0/2898709480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c7410e0c0 msgr2=0x7f8c7419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c7410e0c0 0x7f8c7419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 -- 172.21.15.96:0/2898709480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c74104c70 msgr2=0x7f8c7419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c74104c70 0x7f8c7419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 -- 172.21.15.96:0/2898709480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7419f560 con 0x7f8c7410b6d0 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.027+0000 7f8c788da640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410b6d0 0x7f8c7419ac60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400d950 tx=0x7f8c6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.028+0000 7f8c6a7fc640 1 -- 172.21.15.96:0/2898709480 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64017030 con 0x7f8c7410b6d0 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.028+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7419f850 con 0x7f8c7410b6d0 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.028+0000 7f8c6a7fc640 1 -- 172.21.15.96:0/2898709480 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c64004020 con 0x7f8c7410b6d0 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.028+0000 7f8c6a7fc640 1 -- 172.21.15.96:0/2898709480 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640047a0 con 0x7f8c7410b6d0 2024-04-25T17:12:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.028+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c741a71d0 con 0x7f8c7410b6d0 2024-04-25T17:12:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:42 smithi171 ceph-mon[33189]: pgmap v76: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 645 KiB/s wr, 347 op/s 2024-04-25T17:12:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:42 smithi171 ceph-mon[33189]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T17:12:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:42 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6836/3526163799,v1:172.21.15.97:6837/3526163799] up:boot 2024-04-25T17:12:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:42 smithi171 ceph-mon[33189]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T17:12:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:42 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:12:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:42 smithi171 ceph-mon[33189]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.030+0000 7f8c6a7fc640 1 -- 172.21.15.96:0/2898709480 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8c64004940 con 0x7f8c7410b6d0 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 Processor -- start 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 -- start start 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c6a7fc640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8c600afc50 0x7f8c600b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135991b0 0x565413938af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135702f0 0x565413939030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413939570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56541389f460 con 0x5654135702f0 2024-04-25T17:12:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x565413859590 con 0x5654135991b0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.031+0000 7f8c821f3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56541389b420 con 0x5654135784d0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c798dc640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8c600afc50 0x7f8c600b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c6a7fc640 1 -- 172.21.15.96:0/2898709480 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f8c640dea00 con 0x7f8c7410b6d0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135702f0 0x565413939030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135702f0 0x565413939030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35212/0 (socket says 172.21.15.96:35212) 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 -- 172.21.15.96:0/4278222429 learned_addr learned my addr 172.21.15.96:0/4278222429 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c798dc640 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413939570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c790db640 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135991b0 0x565413938af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 -- 172.21.15.96:0/4278222429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 msgr2=0x565413939570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413939570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 -- 172.21.15.96:0/4278222429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135991b0 msgr2=0x565413938af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135991b0 0x565413938af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 -- 172.21.15.96:0/4278222429 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654138660d0 con 0x5654135702f0 2024-04-25T17:12:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c790db640 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135991b0 0x565413938af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:12:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c798dc640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8c600afc50 0x7f8c600b2160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0094e0 tx=0x7f8c6c02f310 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c788da640 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135702f0 0x565413939030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a2bd0 tx=0x7f8c64005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3697463a30aaf7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c5f7fe640 1 -- 172.21.15.96:0/4278222429 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640a6660 con 0x5654135702f0 2024-04-25T17:12:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c5f7fe640 1 -- 172.21.15.96:0/4278222429 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c640a7250 con 0x5654135702f0 2024-04-25T17:12:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.032+0000 7f8c5f7fe640 1 -- 172.21.15.96:0/4278222429 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640dc380 con 0x5654135702f0 2024-04-25T17:12:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 -- 172.21.15.96:0/4278222429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135702f0 msgr2=0x565413939030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135702f0 0x565413939030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a2bd0 tx=0x7f8c64005dc0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 -- 172.21.15.96:0/4278222429 shutdown_connections 2024-04-25T17:12:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413939570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135702f0 0x565413939030 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 --2- 172.21.15.96:0/4278222429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135991b0 0x565413938af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 -- 172.21.15.96:0/4278222429 >> 172.21.15.96:0/4278222429 conn(0x56541358d300 msgr2=0x5654139352a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 -- 172.21.15.96:0/4278222429 shutdown_connections 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.033+0000 7f8c821f3740 1 -- 172.21.15.96:0/4278222429 wait complete. 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 -- start start 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 Processor -- start 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x565413944b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413945080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413943050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x565413945b60 con 0x5654135991b0 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x565413826b60 con 0x5654135702f0 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565413945ce0 con 0x5654135784d0 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.034+0000 7f8c821f3740 10 client.0 _ll_register_callbacks cb 0x56541384b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413943050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413943050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35226/0 (socket says 172.21.15.96:35226) 2024-04-25T17:12:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 -- 172.21.15.96:0/1234597207 learned_addr learned my addr 172.21.15.96:0/1234597207 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c788da640 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413945080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c790db640 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x565413944b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 -- 172.21.15.96:0/1234597207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 msgr2=0x565413945080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413945080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 -- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 msgr2=0x565413944b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x565413944b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 -- 172.21.15.96:0/1234597207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654135b1210 con 0x5654135991b0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c798dc640 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413943050 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009ca0 tx=0x7f8c6c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6e55a803ba6a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1234597207 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c004500 con 0x5654135991b0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1234597207 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f8c6c0046a0 con 0x5654135991b0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.035+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c640a7250 con 0x5654135991b0 2024-04-25T17:12:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1234597207 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c02f660 con 0x5654135991b0 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 10 client.15549 fetch_fsmap learned FSMap version 20 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56541367d620 con 0x5654135991b0 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1234597207 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c640a7250 con 0x5654135991b0 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1234597207 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8c6c04a040 con 0x5654135991b0 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 10 client.15549 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 20 client.15549 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x565413557d80 0x565413955790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 4 client.15549 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654138ad070 con 0x565413557d80 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.036+0000 7f8c790db640 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x565413557d80 0x565413955790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.037+0000 7f8c5cff9640 10 client.15549 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.037+0000 7f8c790db640 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x565413557d80 0x565413955790 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb2391dc8c25091 server_cookie=7a81361b880c830 in_seq=0 out_seq=0 2024-04-25T17:12:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1234597207 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654138ad070 con 0x565413557d80 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c5cff9640 10 client.15549 handle_command_reply: tid=0 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 client.15549 shutdown 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 msgr2=0x565413943050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413943050 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009ca0 tx=0x7f8c6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 shutdown_connections 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x565413557d80 0x565413955790 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413943050 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135784d0 0x565413945080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1234597207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x565413944b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 >> 172.21.15.96:0/1234597207 conn(0x56541358d300 msgr2=0x56541393a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c5cff9640 0 client.15549 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.038+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 shutdown_connections 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.039+0000 7f8c821f3740 1 -- 172.21.15.96:0/1234597207 wait complete. 2024-04-25T17:12:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.039+0000 7f8c821f3740 20 client.15549 trim_cache size 0 max 16384 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.063+0000 7f8c821f3740 1 Processor -- start 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.063+0000 7f8c821f3740 1 -- start start 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x56541393aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135784d0 0x56541395af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135991b0 0x56541395b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c821f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741adac0 con 0x5654135784d0 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c821f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741b0340 con 0x5654135702f0 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c821f3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741ae130 con 0x5654135991b0 2024-04-25T17:12:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135784d0 0x56541395af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135784d0 0x56541395af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35232/0 (socket says 172.21.15.96:35232) 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 -- 172.21.15.96:0/1236682645 learned_addr learned my addr 172.21.15.96:0/1236682645 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c790db640 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x56541393aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c798dc640 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135991b0 0x56541395b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 -- 172.21.15.96:0/1236682645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135991b0 msgr2=0x56541395b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135991b0 0x56541395b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 -- 172.21.15.96:0/1236682645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 msgr2=0x56541393aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x56541393aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 -- 172.21.15.96:0/1236682645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565413935e70 con 0x5654135784d0 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c788da640 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135784d0 0x56541395af50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a2bd0 tx=0x7f8c640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a50f8310a7df36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.064+0000 7f8c5effd640 1 -- 172.21.15.96:0/1236682645 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640a6660 con 0x5654135784d0 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.065+0000 7f8c5effd640 1 -- 172.21.15.96:0/1236682645 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c640a7250 con 0x5654135784d0 2024-04-25T17:12:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.065+0000 7f8c5effd640 1 -- 172.21.15.96:0/1236682645 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640af9b0 con 0x5654135784d0 2024-04-25T17:12:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 -- 172.21.15.96:0/1236682645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135784d0 msgr2=0x56541395af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135784d0 0x56541395af50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8c640a2bd0 tx=0x7f8c640046c0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 -- 172.21.15.96:0/1236682645 shutdown_connections 2024-04-25T17:12:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135991b0 0x56541395b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135784d0 0x56541395af50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1236682645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135702f0 0x56541393aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 -- 172.21.15.96:0/1236682645 >> 172.21.15.96:0/1236682645 conn(0x56541358d300 msgr2=0x56541393a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 -- 172.21.15.96:0/1236682645 shutdown_connections 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 -- 172.21.15.96:0/1236682645 wait complete. 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 -- start start 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.066+0000 7f8c821f3740 1 Processor -- start 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135702f0 0x56541394b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135784d0 0x565413948860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c821f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413948da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c821f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741ae800 con 0x5654135991b0 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c821f3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741afc70 con 0x5654135784d0 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c821f3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741aeed0 con 0x5654135702f0 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c798dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413948da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c821f3740 10 client.0 _ll_register_callbacks cb 0x7f8c6c005bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c798dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413948da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35238/0 (socket says 172.21.15.96:35238) 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c790db640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135702f0 0x56541394b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c798dc640 1 -- 172.21.15.96:0/1969286008 learned_addr learned my addr 172.21.15.96:0/1969286008 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.067+0000 7f8c788da640 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135784d0 0x565413948860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c798dc640 1 -- 172.21.15.96:0/1969286008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135702f0 msgr2=0x56541394b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c798dc640 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135702f0 0x56541394b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c798dc640 1 -- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135784d0 msgr2=0x565413948860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c798dc640 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135784d0 0x565413948860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c798dc640 1 -- 172.21.15.96:0/1969286008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654139494e0 con 0x5654135991b0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c798dc640 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413948da0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009bf0 tx=0x7f8c6c031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9d4c29cf72f619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1969286008 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c061070 con 0x5654135991b0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1969286008 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f8c6c030af0 con 0x5654135991b0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c640a7250 con 0x5654135991b0 2024-04-25T17:12:43.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1969286008 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c004400 con 0x5654135991b0 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c821f3740 10 client.15555 fetch_fsmap learned FSMap version 20 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5654139bda10 con 0x5654135991b0 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.068+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1969286008 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c640a7250 con 0x5654135991b0 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1969286008 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f8c6c04b070 con 0x5654135991b0 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c821f3740 10 client.15555 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c821f3740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c821f3740 20 client.15555 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56541352d1c0 0x5654139c00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c821f3740 4 client.15555 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654138a6ee0 con 0x56541352d1c0 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c790db640 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56541352d1c0 0x5654139c00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c5cff9640 10 client.15555 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.069+0000 7f8c790db640 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56541352d1c0 0x5654139c00f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f451bb0075b9ee server_cookie=8b5551396f3c4196 in_seq=0 out_seq=0 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.333+0000 7f8c5cff9640 1 -- 172.21.15.96:0/1969286008 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654138a6ee0 con 0x56541352d1c0 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.333+0000 7f8c5cff9640 10 client.15555 handle_command_reply: tid=0 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.333+0000 7f8c821f3740 1 client.15555 shutdown 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 msgr2=0x565413948da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413948da0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009bf0 tx=0x7f8c6c031ba0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 shutdown_connections 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56541352d1c0 0x5654139c00f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654135991b0 0x565413948da0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5654135784d0 0x565413948860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 --2- 172.21.15.96:0/1969286008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654135702f0 0x56541394b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 >> 172.21.15.96:0/1969286008 conn(0x56541358d300 msgr2=0x565413942590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c5cff9640 0 client.15555 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 shutdown_connections 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 1 -- 172.21.15.96:0/1969286008 wait complete. 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.334+0000 7f8c821f3740 20 client.15555 trim_cache size 0 max 16384 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:12:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.335+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8c600afc50 msgr2=0x7f8c600b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.335+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8c600afc50 0x7f8c600b2160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0094e0 tx=0x7f8c6c02f310 comp rx=0 tx=0).stop 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.335+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410b6d0 msgr2=0x7f8c7419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.336+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410b6d0 0x7f8c7419ac60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400d950 tx=0x7f8c6400de10 comp rx=0 tx=0).stop 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.336+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 shutdown_connections 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.336+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c7410e0c0 0x7f8c7419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.336+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c7410b6d0 0x7f8c7419ac60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.336+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8c600afc50 0x7f8c600b2160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.336+0000 7f8c7b33f640 1 --2- 172.21.15.96:0/2898709480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c74104c70 0x7f8c7419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.336+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 >> 172.21.15.96:0/2898709480 conn(0x7f8c740781b0 msgr2=0x7f8c74101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.337+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 shutdown_connections 2024-04-25T17:12:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:43.337+0000 7f8c7b33f640 1 -- 172.21.15.96:0/2898709480 wait complete. 2024-04-25T17:12:43.955 INFO:tasks.workunit.client.0.smithi096.stderr:+ cd .. 2024-04-25T17:12:43.955 INFO:tasks.workunit.client.0.smithi096.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-25T17:12:43.958 INFO:tasks.workunit.client.0.smithi096.stderr:+ OPTIONS=-z 2024-04-25T17:12:43.958 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-25T17:12:43.958 INFO:tasks.workunit.client.0.smithi096.stdout:Seed set to 1 2024-04-25T17:12:44.080 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T17:12:44.080 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T17:12:44.082 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T17:12:44.082 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T17:12:44.082 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T17:12:44.084 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T17:12:44.086 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T17:12:44.087 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T17:12:44.087 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T17:12:44.087 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x81d54 2024-04-25T17:12:44.091 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xb86d7 2024-04-25T17:12:44.098 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xe627e 2024-04-25T17:12:44.137 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0xf3194 2024-04-25T17:12:44.157 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xf3ceb 2024-04-25T17:12:44.164 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0xf47e4 2024-04-25T17:12:44.254 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xfaccc 2024-04-25T17:12:44.726 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xfe423 2024-04-25T17:12:44.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:44 smithi096 ceph-mon[29131]: pgmap v78: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 179 KiB/s wr, 17 op/s 2024-04-25T17:12:44.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:44 smithi097 ceph-mon[32602]: pgmap v78: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 179 KiB/s wr, 17 op/s 2024-04-25T17:12:45.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:44 smithi171 ceph-mon[33189]: pgmap v78: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 179 KiB/s wr, 17 op/s 2024-04-25T17:12:46.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:46 smithi096 ceph-mon[29131]: pgmap v79: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 179 KiB/s wr, 17 op/s 2024-04-25T17:12:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:46 smithi097 ceph-mon[32602]: pgmap v79: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 179 KiB/s wr, 17 op/s 2024-04-25T17:12:46.987 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xfe492 2024-04-25T17:12:47.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:46 smithi171 ceph-mon[33189]: pgmap v79: 97 pgs: 97 active+clean; 28 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 179 KiB/s wr, 17 op/s 2024-04-25T17:12:48.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:48 smithi096 ceph-mon[29131]: pgmap v80: 97 pgs: 97 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-25T17:12:48.939 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xff038 2024-04-25T17:12:48.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:48 smithi097 ceph-mon[32602]: pgmap v80: 97 pgs: 97 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-25T17:12:49.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:48 smithi171 ceph-mon[33189]: pgmap v80: 97 pgs: 97 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-25T17:12:49.807 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0xffcac 2024-04-25T17:12:49.929 INFO:tasks.workunit.client.0.smithi096.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-25T17:12:50.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:50 smithi096 ceph-mon[29131]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-25T17:12:50.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:50 smithi097 ceph-mon[32602]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-25T17:12:51.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:50 smithi171 ceph-mon[33189]: pgmap v81: 97 pgs: 97 active+clean; 39 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-04-25T17:12:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:52 smithi096 ceph-mon[29131]: pgmap v82: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 734 op/s 2024-04-25T17:12:52.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:52 smithi097 ceph-mon[32602]: pgmap v82: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 734 op/s 2024-04-25T17:12:53.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:52 smithi171 ceph-mon[33189]: pgmap v82: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 734 op/s 2024-04-25T17:12:54.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:54 smithi096 ceph-mon[29131]: pgmap v83: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 649 op/s 2024-04-25T17:12:54.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:54 smithi097 ceph-mon[32602]: pgmap v83: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 649 op/s 2024-04-25T17:12:55.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:54 smithi171 ceph-mon[33189]: pgmap v83: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 649 op/s 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 -- 172.21.15.96:0/3366986758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810e110 msgr2=0x7fa3981105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 --2- 172.21.15.96:0/3366986758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810e110 0x7fa3981105e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa390009ca0 tx=0x7fa39002a950 comp rx=0 tx=0).stop 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 -- 172.21.15.96:0/3366986758 shutdown_connections 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 --2- 172.21.15.96:0/3366986758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810e110 0x7fa3981105e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 --2- 172.21.15.96:0/3366986758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa39810b760 0x7fa39810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 --2- 172.21.15.96:0/3366986758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa398104d00 0x7fa398105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 -- 172.21.15.96:0/3366986758 >> 172.21.15.96:0/3366986758 conn(0x7fa398079170 msgr2=0x7fa3980795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 -- 172.21.15.96:0/3366986758 shutdown_connections 2024-04-25T17:12:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.443+0000 7fa39f014640 1 -- 172.21.15.96:0/3366986758 wait complete. 2024-04-25T17:12:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 Processor -- start 2024-04-25T17:12:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 -- start start 2024-04-25T17:12:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa398104d00 0x7fa3981025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810b760 0x7fa398102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa39810e110 0x7fa398103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa398113d50 con 0x7fa39810b760 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa398113bd0 con 0x7fa39810e110 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39f014640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa398113ed0 con 0x7fa398104d00 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810b760 0x7fa398102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39cdb0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa398104d00 0x7fa3981025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810b760 0x7fa398102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41424/0 (socket says 172.21.15.96:41424) 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 -- 172.21.15.96:0/32889177 learned_addr learned my addr 172.21.15.96:0/32889177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa39d5b1640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa39810e110 0x7fa398103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 -- 172.21.15.96:0/32889177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa398104d00 msgr2=0x7fa3981025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa398104d00 0x7fa3981025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 -- 172.21.15.96:0/32889177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa39810e110 msgr2=0x7fa398103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa39810e110 0x7fa398103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 -- 172.21.15.96:0/32889177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3981a8a30 con 0x7fa39810b760 2024-04-25T17:12:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.444+0000 7fa397fff640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810b760 0x7fa398102b00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00e9f0 tx=0x7fa38c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.445+0000 7fa395ffb640 1 -- 172.21.15.96:0/32889177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00cd80 con 0x7fa39810b760 2024-04-25T17:12:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.445+0000 7fa395ffb640 1 -- 172.21.15.96:0/32889177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38c0054d0 con 0x7fa39810b760 2024-04-25T17:12:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.445+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3981a8d20 con 0x7fa39810b760 2024-04-25T17:12:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.445+0000 7fa395ffb640 1 -- 172.21.15.96:0/32889177 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c010690 con 0x7fa39810b760 2024-04-25T17:12:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.445+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3981a91c0 con 0x7fa39810b760 2024-04-25T17:12:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.447+0000 7fa395ffb640 1 -- 172.21.15.96:0/32889177 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa38c004e90 con 0x7fa39810b760 2024-04-25T17:12:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.447+0000 7fa3a5f9b740 1 Processor -- start 2024-04-25T17:12:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.447+0000 7fa3a5f9b740 1 -- start start 2024-04-25T17:12:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa395ffb640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa3800afc50 0x7fa3800b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cfd5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfd6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55999cf3c460 con 0x55999cc154d0 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55999cef6590 con 0x55999cc361b0 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55999cf38420 con 0x55999cc0d2f0 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39d5b1640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa3800afc50 0x7fa3800b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cfd5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cfd5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:47996/0 (socket says 172.21.15.96:47996) 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa395ffb640 1 -- 172.21.15.96:0/32889177 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fa38c0df380 con 0x7fa39810b760 2024-04-25T17:12:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 -- 172.21.15.96:0/48078031 learned_addr learned my addr 172.21.15.96:0/48078031 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa397fff640 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfd6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39d5b1640 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 -- 172.21.15.96:0/48078031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 msgr2=0x55999cfd6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfd6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 -- 172.21.15.96:0/48078031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 msgr2=0x55999cfd6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 -- 172.21.15.96:0/48078031 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999cf030d0 con 0x55999cc361b0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39d5b1640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa3800afc50 0x7fa3800b2160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa390005e30 tx=0x7fa390009a00 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.448+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cfd5af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa388009cf0 tx=0x7fa388011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b52e5a9d17dc56a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.449+0000 7fa37effd640 1 -- 172.21.15.96:0/48078031 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38801e070 con 0x55999cc361b0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.449+0000 7fa37effd640 1 -- 172.21.15.96:0/48078031 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa388019ae0 con 0x55999cc361b0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.449+0000 7fa37effd640 1 -- 172.21.15.96:0/48078031 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38800e970 con 0x55999cc361b0 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.449+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/48078031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 msgr2=0x55999cfd5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.449+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cfd5af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa388009cf0 tx=0x7fa388011810 comp rx=0 tx=0).stop 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.449+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/48078031 shutdown_connections 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.449+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfd6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/48078031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cfd5af0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/48078031 >> 172.21.15.96:0/48078031 conn(0x55999cc2a300 msgr2=0x55999cfd22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/48078031 shutdown_connections 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/48078031 wait complete. 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 -- start start 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 Processor -- start 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc0d2f0 0x55999cfe01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfe06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999cfe6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55999cfe2c20 con 0x55999cc154d0 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55999cec3b60 con 0x55999cc0d2f0 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55999cfe2da0 con 0x55999cc361b0 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.450+0000 7fa3a5f9b740 10 client.0 _ll_register_callbacks cb 0x55999cee82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfe06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc0d2f0 0x55999cfe01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfe06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41446/0 (socket says 172.21.15.96:41446) 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa397fff640 1 -- 172.21.15.96:0/604277438 learned_addr learned my addr 172.21.15.96:0/604277438 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc0d2f0 0x55999cfe01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:48004/0 (socket says 172.21.15.96:48004) 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39d5b1640 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999cfe6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 -- 172.21.15.96:0/604277438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 msgr2=0x55999cfe6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999cfe6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.462 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 -- 172.21.15.96:0/604277438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 msgr2=0x55999cfe06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfe06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 -- 172.21.15.96:0/604277438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999cc4e210 con 0x55999cc0d2f0 2024-04-25T17:12:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa397fff640 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfe06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:12:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc0d2f0 0x55999cfe01b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa38801c040 tx=0x7fa388019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2e0864519ba540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa377fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T17:12:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa377fff640 1 -- 172.21.15.96:0/604277438 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38801e050 con 0x55999cc0d2f0 2024-04-25T17:12:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa377fff640 1 -- 172.21.15.96:0/604277438 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fa388005240 con 0x55999cc0d2f0 2024-04-25T17:12:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa388019ae0 con 0x55999cc0d2f0 2024-04-25T17:12:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa377fff640 1 -- 172.21.15.96:0/604277438 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa388033050 con 0x55999cc0d2f0 2024-04-25T17:12:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa3a5f9b740 10 client.25178 fetch_fsmap learned FSMap version 20 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.451+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55999cd1a620 con 0x55999cc0d2f0 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa377fff640 1 -- 172.21.15.96:0/604277438 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa388019ae0 con 0x55999cc0d2f0 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa377fff640 1 -- 172.21.15.96:0/604277438 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa388004c00 con 0x55999cc0d2f0 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa3a5f9b740 10 client.25178 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa3a5f9b740 10 client.25178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa3a5f9b740 20 client.25178 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbf4d80 0x55999cff2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa3a5f9b740 4 client.25178 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55999cf4a070 con 0x55999cbf4d80 2024-04-25T17:12:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa397fff640 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbf4d80 0x55999cff2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa377fff640 10 client.25178 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.452+0000 7fa397fff640 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbf4d80 0x55999cff2660 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240f918663f8ce6 server_cookie=9c9c5c41486c259 in_seq=0 out_seq=0 2024-04-25T17:12:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa377fff640 1 -- 172.21.15.96:0/604277438 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55999cf4a070 con 0x55999cbf4d80 2024-04-25T17:12:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa377fff640 10 client.25178 handle_command_reply: tid=0 2024-04-25T17:12:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 client.25178 shutdown 2024-04-25T17:12:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc0d2f0 msgr2=0x55999cfe01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc0d2f0 0x55999cfe01b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa38801c040 tx=0x7fa388019660 comp rx=0 tx=0).stop 2024-04-25T17:12:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 shutdown_connections 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbf4d80 0x55999cff2660 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999cfe6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfe06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/604277438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc0d2f0 0x55999cfe01b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa377fff640 0 client.25178 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 >> 172.21.15.96:0/604277438 conn(0x55999cc2a300 msgr2=0x55999cfd5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 shutdown_connections 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/604277438 wait complete. 2024-04-25T17:12:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.453+0000 7fa3a5f9b740 20 client.25178 trim_cache size 0 max 16384 2024-04-25T17:12:56.478 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 Processor -- start 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 -- start start 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfe6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cff7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981afbb0 con 0x55999cc154d0 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b2430 con 0x55999cc361b0 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b0220 con 0x55999cc0d2f0 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41456/0 (socket says 172.21.15.96:41456) 2024-04-25T17:12:56.479 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 -- 172.21.15.96:0/1506561326 learned_addr learned my addr 172.21.15.96:0/1506561326 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa39d5b1640 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cff7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfe6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 -- 172.21.15.96:0/1506561326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 msgr2=0x55999cfe6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfe6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 -- 172.21.15.96:0/1506561326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 msgr2=0x55999cff7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cff7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 -- 172.21.15.96:0/1506561326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999cfde4f0 con 0x55999cc154d0 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.469+0000 7fa397fff640 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd2370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa38c004a40 tx=0x7fa38c009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf2cd86535b9ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa37e7fc640 1 -- 172.21.15.96:0/1506561326 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0c1e20 con 0x55999cc154d0 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa37e7fc640 1 -- 172.21.15.96:0/1506561326 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38c0a7040 con 0x55999cc154d0 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa37e7fc640 1 -- 172.21.15.96:0/1506561326 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0a33f0 con 0x55999cc154d0 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1506561326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 msgr2=0x55999cfd2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd2370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa38c004a40 tx=0x7fa38c009330 comp rx=0 tx=0).stop 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1506561326 shutdown_connections 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc361b0 0x55999cff7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc154d0 0x55999cfd2370 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1506561326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc0d2f0 0x55999cfe6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1506561326 >> 172.21.15.96:0/1506561326 conn(0x55999cc2a300 msgr2=0x55999cfd1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1506561326 shutdown_connections 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.470+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1506561326 wait complete. 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 -- start start 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 Processor -- start 2024-04-25T17:12:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc0d2f0 0x55999cfe78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc154d0 0x55999cfe7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999d052830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b08f0 con 0x55999cc0d2f0 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b1d60 con 0x55999cc154d0 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa3981b0fc0 con 0x55999cc361b0 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa3a5f9b740 10 client.0 _ll_register_callbacks cb 0x7fa38801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa39cdb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc0d2f0 0x55999cfe78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa39cdb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc0d2f0 0x55999cfe78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41472/0 (socket says 172.21.15.96:41472) 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa39cdb0640 1 -- 172.21.15.96:0/1776286688 learned_addr learned my addr 172.21.15.96:0/1776286688 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa39d5b1640 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999d052830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.471+0000 7fa397fff640 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc154d0 0x55999cfe7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa39cdb0640 1 -- 172.21.15.96:0/1776286688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 msgr2=0x55999d052830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999d052830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa39cdb0640 1 -- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc154d0 msgr2=0x55999cfe7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc154d0 0x55999cfe7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa39cdb0640 1 -- 172.21.15.96:0/1776286688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999d052d70 con 0x55999cc0d2f0 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa39cdb0640 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc0d2f0 0x55999cfe78f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa3880097e0 tx=0x7fa388009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f237df689f5699 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa377fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa377fff640 1 -- 172.21.15.96:0/1776286688 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38803d060 con 0x55999cc0d2f0 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa377fff640 1 -- 172.21.15.96:0/1776286688 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fa38800e3b0 con 0x55999cc0d2f0 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa38c0a7040 con 0x55999cc0d2f0 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa377fff640 1 -- 172.21.15.96:0/1776286688 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa38801e040 con 0x55999cc0d2f0 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa3a5f9b740 10 client.15573 fetch_fsmap learned FSMap version 20 2024-04-25T17:12:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55999d05a090 con 0x55999cc0d2f0 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.472+0000 7fa377fff640 1 -- 172.21.15.96:0/1776286688 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38c0a7040 con 0x55999cc0d2f0 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa377fff640 1 -- 172.21.15.96:0/1776286688 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa3880035f0 con 0x55999cc0d2f0 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa3a5f9b740 10 client.15573 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa3a5f9b740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa3a5f9b740 20 client.15573 populate_metadata read hostname 'smithi096' 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbca1c0 0x55999d05c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa3a5f9b740 4 client.15573 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55999cf43ee0 con 0x55999cbca1c0 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa397fff640 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbca1c0 0x55999d05c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa377fff640 10 client.15573 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.473+0000 7fa397fff640 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbca1c0 0x55999d05c7f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fe239313ab3a1f server_cookie=33c589930c80b5e5 in_seq=0 out_seq=0 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa377fff640 1 -- 172.21.15.96:0/1776286688 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55999cf43ee0 con 0x55999cbca1c0 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa377fff640 10 client.15573 handle_command_reply: tid=0 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 client.15573 shutdown 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc0d2f0 msgr2=0x55999cfe78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc0d2f0 0x55999cfe78f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa3880097e0 tx=0x7fa388009cb0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 shutdown_connections 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55999cbca1c0 0x55999d05c7f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55999cc361b0 0x55999d052830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55999cc154d0 0x55999cfe7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa377fff640 0 client.15573 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 --2- 172.21.15.96:0/1776286688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55999cc0d2f0 0x55999cfe78f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 >> 172.21.15.96:0/1776286688 conn(0x55999cc2a300 msgr2=0x55999cfd5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 shutdown_connections 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.512+0000 7fa3a5f9b740 1 -- 172.21.15.96:0/1776286688 wait complete. 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.513+0000 7fa3a5f9b740 20 client.15573 trim_cache size 0 max 16384 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:12:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:12:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:12:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.513+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa3800afc50 msgr2=0x7fa3800b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.513+0000 7fa39f014640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa3800afc50 0x7fa3800b2160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa390005e30 tx=0x7fa390009a00 comp rx=0 tx=0).stop 2024-04-25T17:12:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.513+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810b760 msgr2=0x7fa398102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.513+0000 7fa39f014640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810b760 0x7fa398102b00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00e9f0 tx=0x7fa38c00edf0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 shutdown_connections 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa3800afc50 0x7fa3800b2160 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa39810e110 0x7fa398103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa39810b760 0x7fa398102b00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 --2- 172.21.15.96:0/32889177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa398104d00 0x7fa3981025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 >> 172.21.15.96:0/32889177 conn(0x7fa398079170 msgr2=0x7fa398101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 shutdown_connections 2024-04-25T17:12:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:12:56.514+0000 7fa39f014640 1 -- 172.21.15.96:0/32889177 wait complete. 2024-04-25T17:12:56.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:56 smithi096 ceph-mon[29131]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 612 op/s 2024-04-25T17:12:56.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:56 smithi097 ceph-mon[32602]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 612 op/s 2024-04-25T17:12:57.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:56 smithi171 ceph-mon[33189]: pgmap v84: 97 pgs: 97 active+clean; 43 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 612 op/s 2024-04-25T17:12:58.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:12:58 smithi096 ceph-mon[29131]: pgmap v85: 97 pgs: 97 active+clean; 45 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 897 op/s 2024-04-25T17:12:58.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:12:58 smithi097 ceph-mon[32602]: pgmap v85: 97 pgs: 97 active+clean; 45 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 897 op/s 2024-04-25T17:12:59.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:12:58 smithi171 ceph-mon[33189]: pgmap v85: 97 pgs: 97 active+clean; 45 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 897 op/s 2024-04-25T17:13:00.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:00 smithi096 ceph-mon[29131]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 612 op/s 2024-04-25T17:13:00.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:00 smithi097 ceph-mon[32602]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 612 op/s 2024-04-25T17:13:01.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:00 smithi171 ceph-mon[33189]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 612 op/s 2024-04-25T17:13:02.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:02 smithi171 ceph-mon[33189]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 889 op/s 2024-04-25T17:13:02.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:02 smithi096 ceph-mon[29131]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 889 op/s 2024-04-25T17:13:02.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:02 smithi097 ceph-mon[32602]: pgmap v87: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.5 MiB/s wr, 889 op/s 2024-04-25T17:13:04.902 INFO:tasks.workunit.client.0.smithi096.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-25T17:13:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:04 smithi096 ceph-mon[29131]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 562 op/s 2024-04-25T17:13:04.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:04 smithi097 ceph-mon[32602]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 562 op/s 2024-04-25T17:13:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:04 smithi171 ceph-mon[33189]: pgmap v88: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 562 op/s 2024-04-25T17:13:05.279 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xffcdd 2024-04-25T17:13:06.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:06 smithi096 ceph-mon[29131]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 562 op/s 2024-04-25T17:13:06.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:06 smithi097 ceph-mon[32602]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 562 op/s 2024-04-25T17:13:07.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:06 smithi171 ceph-mon[33189]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 562 op/s 2024-04-25T17:13:07.104 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xffd97 2024-04-25T17:13:08.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:08 smithi096 ceph-mon[29131]: pgmap v90: 97 pgs: 97 active+clean; 40 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 800 op/s 2024-04-25T17:13:08.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:08 smithi097 ceph-mon[32602]: pgmap v90: 97 pgs: 97 active+clean; 40 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 800 op/s 2024-04-25T17:13:09.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:08 smithi171 ceph-mon[33189]: pgmap v90: 97 pgs: 97 active+clean; 40 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 800 op/s 2024-04-25T17:13:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 -- 172.21.15.96:0/450830614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108f30 msgr2=0x7f7a28102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/450830614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108f30 0x7f7a28102b70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009900 tx=0x7f7a1c02a410 comp rx=0 tx=0).stop 2024-04-25T17:13:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 -- 172.21.15.96:0/450830614 shutdown_connections 2024-04-25T17:13:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/450830614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281031e0 0x7f7a2810f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/450830614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108f30 0x7f7a28102b70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/450830614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28108a90 0x7f7a28102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 -- 172.21.15.96:0/450830614 >> 172.21.15.96:0/450830614 conn(0x7f7a280fe260 msgr2=0x7f7a28100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 -- 172.21.15.96:0/450830614 shutdown_connections 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.615+0000 7f7a2d86e640 1 -- 172.21.15.96:0/450830614 wait complete. 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 Processor -- start 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 -- start start 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281031e0 0x7f7a28199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108a90 0x7f7a28199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28108f30 0x7f7a2819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28112c80 con 0x7f7a28108a90 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28112b00 con 0x7f7a28108f30 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a2d86e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a28112e00 con 0x7f7a281031e0 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a267fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108a90 0x7f7a28199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a26ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281031e0 0x7f7a28199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a267fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108a90 0x7f7a28199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46768/0 (socket says 172.21.15.96:46768) 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.616+0000 7f7a267fc640 1 -- 172.21.15.96:0/3369032590 learned_addr learned my addr 172.21.15.96:0/3369032590 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a277fe640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28108f30 0x7f7a2819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a26ffd640 1 -- 172.21.15.96:0/3369032590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28108f30 msgr2=0x7f7a2819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a26ffd640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28108f30 0x7f7a2819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a26ffd640 1 -- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108a90 msgr2=0x7f7a28199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a26ffd640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108a90 0x7f7a28199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a26ffd640 1 -- 172.21.15.96:0/3369032590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2819e580 con 0x7f7a281031e0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a267fc640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108a90 0x7f7a28199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a26ffd640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281031e0 0x7f7a28199720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800ead0 tx=0x7f7a1800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a2c86c640 1 -- 172.21.15.96:0/3369032590 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1801c070 con 0x7f7a281031e0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2819e870 con 0x7f7a281031e0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a2c86c640 1 -- 172.21.15.96:0/3369032590 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a1800bb70 con 0x7f7a281031e0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a2c86c640 1 -- 172.21.15.96:0/3369032590 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18016610 con 0x7f7a281031e0 2024-04-25T17:13:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.617+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a28074dd0 con 0x7f7a281031e0 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 Processor -- start 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 -- start start 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x56353973faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393772f0 0x563539740030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539740570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5635396a6460 con 0x5635393772f0 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563539660590 con 0x5635393a01b0 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.620+0000 7f7a34803740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5635396a2420 con 0x56353937f4d0 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x56353973faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a2c86c640 1 -- 172.21.15.96:0/3369032590 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7a1801f070 con 0x7f7a281031e0 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x56353973faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:56758/0 (socket says 172.21.15.96:56758) 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 -- 172.21.15.96:0/1536480462 learned_addr learned my addr 172.21.15.96:0/1536480462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a267fc640 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539740570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a26ffd640 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393772f0 0x563539740030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 -- 172.21.15.96:0/1536480462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 msgr2=0x563539740570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539740570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 -- 172.21.15.96:0/1536480462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393772f0 msgr2=0x563539740030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393772f0 0x563539740030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 -- 172.21.15.96:0/1536480462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56353966d0d0 con 0x5635393a01b0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a277fe640 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x56353973faf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009fd0 tx=0x7f7a200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5173eef1df03b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a0dffb640 1 -- 172.21.15.96:0/1536480462 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20004d20 con 0x5635393a01b0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a2c86c640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a100afc50 0x7f7a100b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a0dffb640 1 -- 172.21.15.96:0/1536480462 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a20004050 con 0x5635393a01b0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a0dffb640 1 -- 172.21.15.96:0/1536480462 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20005280 con 0x5635393a01b0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.621+0000 7f7a26ffd640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a100afc50 0x7f7a100b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a2c86c640 1 -- 172.21.15.96:0/3369032590 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f7a180e05d0 con 0x7f7a281031e0 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 -- 172.21.15.96:0/1536480462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 msgr2=0x56353973faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a26ffd640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a100afc50 0x7f7a100b2160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a180ddde0 tx=0x7f7a18009700 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x56353973faf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009fd0 tx=0x7f7a200118d0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 -- 172.21.15.96:0/1536480462 shutdown_connections 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539740570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393772f0 0x563539740030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 --2- 172.21.15.96:0/1536480462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x56353973faf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 -- 172.21.15.96:0/1536480462 >> 172.21.15.96:0/1536480462 conn(0x563539394300 msgr2=0x56353973c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 -- 172.21.15.96:0/1536480462 shutdown_connections 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 -- 172.21.15.96:0/1536480462 wait complete. 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.622+0000 7f7a34803740 1 -- start start 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 1 Processor -- start 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x56353974bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x56353974c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539749ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56353974cc20 con 0x56353937f4d0 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56353962db60 con 0x5635393a01b0 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56353974cda0 con 0x5635393772f0 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a34803740 10 client.0 _ll_register_callbacks cb 0x5635396522b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a267fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x56353974c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a267fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x56353974c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46786/0 (socket says 172.21.15.96:46786) 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a267fc640 1 -- 172.21.15.96:0/2056020169 learned_addr learned my addr 172.21.15.96:0/2056020169 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a277fe640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x56353974bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a277fe640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539749ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a267fc640 1 -- 172.21.15.96:0/2056020169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 msgr2=0x56353974bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a267fc640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x56353974bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a267fc640 1 -- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 msgr2=0x563539749ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.623+0000 7f7a267fc640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539749ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a267fc640 1 -- 172.21.15.96:0/2056020169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635393b8210 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a277fe640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539749ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a267fc640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x56353974c040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006010 tx=0x7f7a1c031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c386edf890ed6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a06ffd640 1 -- 172.21.15.96:0/2056020169 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c035040 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a06ffd640 1 -- 172.21.15.96:0/2056020169 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f7a1c004450 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a20004050 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a06ffd640 1 -- 172.21.15.96:0/2056020169 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c005020 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 10 client.15585 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563539484620 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a06ffd640 1 -- 172.21.15.96:0/2056020169 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a20004050 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a06ffd640 1 -- 172.21.15.96:0/2056020169 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f7a1c0035f0 con 0x56353937f4d0 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 10 client.15585 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 20 client.15585 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56353935ed80 0x56353975c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 4 client.15585 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.624+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635396b4070 con 0x56353935ed80 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.625+0000 7f7a277fe640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56353935ed80 0x56353975c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.625+0000 7f7a06ffd640 10 client.15585 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.625+0000 7f7a277fe640 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56353935ed80 0x56353975c6d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e58c210311b553 server_cookie=1acb7ff6614e7de4 in_seq=0 out_seq=0 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a06ffd640 1 -- 172.21.15.96:0/2056020169 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5635396b4070 con 0x56353935ed80 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a06ffd640 10 client.15585 handle_command_reply: tid=0 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 client.15585 shutdown 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 msgr2=0x56353974c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x56353974c040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006010 tx=0x7f7a1c031740 comp rx=0 tx=0).stop 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 shutdown_connections 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56353935ed80 0x56353975c6d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539749ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x56353974c040 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 --2- 172.21.15.96:0/2056020169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x56353974bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a06ffd640 0 client.15585 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 >> 172.21.15.96:0/2056020169 conn(0x563539394300 msgr2=0x56353973f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 shutdown_connections 2024-04-25T17:13:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 1 -- 172.21.15.96:0/2056020169 wait complete. 2024-04-25T17:13:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.626+0000 7f7a34803740 20 client.15585 trim_cache size 0 max 16384 2024-04-25T17:13:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 Processor -- start 2024-04-25T17:13:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 -- start start 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393772f0 0x563539761ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539762410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393a01b0 0x563539752030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281ab800 con 0x5635393a01b0 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281ae080 con 0x5635393772f0 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a34803740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281abe70 con 0x56353937f4d0 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393a01b0 0x563539752030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393a01b0 0x563539752030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46798/0 (socket says 172.21.15.96:46798) 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a277fe640 1 -- 172.21.15.96:0/1509895183 learned_addr learned my addr 172.21.15.96:0/1509895183 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a267fc640 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539762410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.642+0000 7f7a277fe640 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393772f0 0x563539761ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a277fe640 1 -- 172.21.15.96:0/1509895183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 msgr2=0x563539762410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a277fe640 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539762410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a277fe640 1 -- 172.21.15.96:0/1509895183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393772f0 msgr2=0x563539761ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a277fe640 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393772f0 0x563539761ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a277fe640 1 -- 172.21.15.96:0/1509895183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563539752570 con 0x5635393a01b0 2024-04-25T17:13:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a277fe640 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393a01b0 0x563539752030 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7a20003ac0 tx=0x7f7a20009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b0e4eeaa4b93b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a0d7fa640 1 -- 172.21.15.96:0/1509895183 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20004050 con 0x5635393a01b0 2024-04-25T17:13:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a0d7fa640 1 -- 172.21.15.96:0/1509895183 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a200046a0 con 0x5635393a01b0 2024-04-25T17:13:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.643+0000 7f7a0d7fa640 1 -- 172.21.15.96:0/1509895183 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000fb00 con 0x5635393a01b0 2024-04-25T17:13:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 -- 172.21.15.96:0/1509895183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393a01b0 msgr2=0x563539752030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393a01b0 0x563539752030 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7a20003ac0 tx=0x7f7a20009da0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 -- 172.21.15.96:0/1509895183 shutdown_connections 2024-04-25T17:13:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635393a01b0 0x563539752030 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56353937f4d0 0x563539762410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 --2- 172.21.15.96:0/1509895183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393772f0 0x563539761ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 -- 172.21.15.96:0/1509895183 >> 172.21.15.96:0/1509895183 conn(0x563539394300 msgr2=0x5635397488c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 -- 172.21.15.96:0/1509895183 shutdown_connections 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 -- 172.21.15.96:0/1509895183 wait complete. 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 -- start start 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.644+0000 7f7a34803740 1 Processor -- start 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a34803740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x5635397bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a34803740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x5635397bd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a34803740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539751020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a34803740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281ac540 con 0x56353937f4d0 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a34803740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281ad9b0 con 0x5635393a01b0 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a34803740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281acc10 con 0x5635393772f0 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a34803740 10 client.0 _ll_register_callbacks cb 0x7f7a1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x5635397bd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x5635397bd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46810/0 (socket says 172.21.15.96:46810) 2024-04-25T17:13:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a277fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x5635397bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 -- 172.21.15.96:0/4219005844 learned_addr learned my addr 172.21.15.96:0/4219005844 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a277fe640 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539751020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 -- 172.21.15.96:0/4219005844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 msgr2=0x5635397bd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x5635397bd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 -- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 msgr2=0x563539751020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539751020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 -- 172.21.15.96:0/4219005844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563539751660 con 0x56353937f4d0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a277fe640 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x5635397bd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a267fc640 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x5635397bd6e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x563539736520 tx=0x7f7a1c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb869b84b696ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.645+0000 7f7a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a06ffd640 1 -- 172.21.15.96:0/4219005844 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c035070 con 0x56353937f4d0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a06ffd640 1 -- 172.21.15.96:0/4219005844 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f7a1c0051c0 con 0x56353937f4d0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a200046a0 con 0x56353937f4d0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a06ffd640 1 -- 172.21.15.96:0/4219005844 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c0053c0 con 0x56353937f4d0 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 10 client.15591 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5635397c4780 con 0x56353937f4d0 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a06ffd640 1 -- 172.21.15.96:0/4219005844 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a200046a0 con 0x56353937f4d0 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a06ffd640 1 -- 172.21.15.96:0/4219005844 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f7a1c058030 con 0x56353937f4d0 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 10 client.15591 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 20 client.15591 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5635393341c0 0x5635397c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 4 client.15591 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635396adee0 con 0x5635393341c0 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.646+0000 7f7a277fe640 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5635393341c0 0x5635397c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.647+0000 7f7a06ffd640 10 client.15591 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.647+0000 7f7a277fe640 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5635393341c0 0x5635397c6f10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d36224a6533d1b7 server_cookie=1e497f775d795fc3 in_seq=0 out_seq=0 2024-04-25T17:13:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a06ffd640 1 -- 172.21.15.96:0/4219005844 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635396adee0 con 0x5635393341c0 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a06ffd640 10 client.15591 handle_command_reply: tid=0 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 client.15591 shutdown 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 msgr2=0x5635397bd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x5635397bd6e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x563539736520 tx=0x7f7a1c000bc0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 shutdown_connections 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5635393341c0 0x5635397c6f10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635393a01b0 0x563539751020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56353937f4d0 0x5635397bd6e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 --2- 172.21.15.96:0/4219005844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5635393772f0 0x5635397bd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a06ffd640 0 client.15591 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 >> 172.21.15.96:0/4219005844 conn(0x563539394300 msgr2=0x5635397437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 shutdown_connections 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 1 -- 172.21.15.96:0/4219005844 wait complete. 2024-04-25T17:13:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.663+0000 7f7a34803740 20 client.15591 trim_cache size 0 max 16384 2024-04-25T17:13:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:13:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:13:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:13:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a100afc50 msgr2=0x7f7a100b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a100afc50 0x7f7a100b2160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a180ddde0 tx=0x7f7a18009700 comp rx=0 tx=0).stop 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281031e0 msgr2=0x7f7a28199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281031e0 0x7f7a28199720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800ead0 tx=0x7f7a1800ef90 comp rx=0 tx=0).stop 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 shutdown_connections 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a100afc50 0x7f7a100b2160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28108f30 0x7f7a2819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a28108a90 0x7f7a28199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 --2- 172.21.15.96:0/3369032590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a281031e0 0x7f7a28199720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 >> 172.21.15.96:0/3369032590 conn(0x7f7a280fe260 msgr2=0x7f7a2810e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 shutdown_connections 2024-04-25T17:13:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:09.664+0000 7f7a2d86e640 1 -- 172.21.15.96:0/3369032590 wait complete. 2024-04-25T17:13:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:10 smithi096 ceph-mon[29131]: pgmap v91: 97 pgs: 97 active+clean; 40 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 514 op/s 2024-04-25T17:13:10.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:10 smithi097 ceph-mon[32602]: pgmap v91: 97 pgs: 97 active+clean; 40 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 514 op/s 2024-04-25T17:13:11.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:10 smithi171 ceph-mon[33189]: pgmap v91: 97 pgs: 97 active+clean; 40 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 514 op/s 2024-04-25T17:13:12.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:12 smithi096 ceph-mon[29131]: pgmap v92: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 739 op/s 2024-04-25T17:13:12.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:12 smithi097 ceph-mon[32602]: pgmap v92: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 739 op/s 2024-04-25T17:13:13.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:12 smithi171 ceph-mon[33189]: pgmap v92: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 739 op/s 2024-04-25T17:13:14.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:14 smithi096 ceph-mon[29131]: pgmap v93: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 461 op/s 2024-04-25T17:13:14.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:14 smithi097 ceph-mon[32602]: pgmap v93: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 461 op/s 2024-04-25T17:13:15.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:14 smithi171 ceph-mon[33189]: pgmap v93: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 461 op/s 2024-04-25T17:13:15.809 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xffdb8 2024-04-25T17:13:16.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:16 smithi096 ceph-mon[29131]: pgmap v94: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 461 op/s 2024-04-25T17:13:16.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:16 smithi097 ceph-mon[32602]: pgmap v94: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 461 op/s 2024-04-25T17:13:17.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:16 smithi171 ceph-mon[33189]: pgmap v94: 97 pgs: 97 active+clean; 35 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 461 op/s 2024-04-25T17:13:18.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:18 smithi096 ceph-mon[29131]: pgmap v95: 97 pgs: 97 active+clean; 37 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 661 op/s 2024-04-25T17:13:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:18 smithi097 ceph-mon[32602]: pgmap v95: 97 pgs: 97 active+clean; 37 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 661 op/s 2024-04-25T17:13:19.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:18 smithi171 ceph-mon[33189]: pgmap v95: 97 pgs: 97 active+clean; 37 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 661 op/s 2024-04-25T17:13:20.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:20 smithi096 ceph-mon[29131]: pgmap v96: 97 pgs: 97 active+clean; 37 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2024-04-25T17:13:20.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:20 smithi097 ceph-mon[32602]: pgmap v96: 97 pgs: 97 active+clean; 37 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2024-04-25T17:13:21.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:20 smithi171 ceph-mon[33189]: pgmap v96: 97 pgs: 97 active+clean; 37 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2024-04-25T17:13:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/1592155756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d0108a60 msgr2=0x7fd7d0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/1592155756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d0108a60 0x7fd7d0102610 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009900 tx=0x7fd7cc02a410 comp rx=0 tx=0).stop 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/1592155756 shutdown_connections 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/1592155756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d01031c0 0x7fd7d010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/1592155756 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7d0108f00 0x7fd7d0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/1592155756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d0108a60 0x7fd7d0102610 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/1592155756 >> 172.21.15.96:0/1592155756 conn(0x7fd7d00781b0 msgr2=0x7fd7d00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/1592155756 shutdown_connections 2024-04-25T17:13:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.762+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/1592155756 wait complete. 2024-04-25T17:13:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 Processor -- start 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 -- start start 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d01031c0 0x7fd7d0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d0108a60 0x7fd7d0199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7d0108f00 0x7fd7d019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0112c60 con 0x7fd7d01031c0 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0112ae0 con 0x7fd7d0108a60 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d7fb4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0112de0 con 0x7fd7d0108f00 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d5d50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d01031c0 0x7fd7d0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d5d50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d01031c0 0x7fd7d0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41684/0 (socket says 172.21.15.96:41684) 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d5d50640 1 -- 172.21.15.96:0/995197130 learned_addr learned my addr 172.21.15.96:0/995197130 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d554f640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d0108a60 0x7fd7d0199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.763+0000 7fd7d6551640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7d0108f00 0x7fd7d019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d5d50640 1 -- 172.21.15.96:0/995197130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7d0108f00 msgr2=0x7fd7d019de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7d0108f00 0x7fd7d019de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d5d50640 1 -- 172.21.15.96:0/995197130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d0108a60 msgr2=0x7fd7d0199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d0108a60 0x7fd7d0199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d5d50640 1 -- 172.21.15.96:0/995197130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d019e4a0 con 0x7fd7d01031c0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d01031c0 0x7fd7d0199640 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc02a930 tx=0x7fd7cc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7c6ffd640 1 -- 172.21.15.96:0/995197130 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc03f030 con 0x7fd7d01031c0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7c6ffd640 1 -- 172.21.15.96:0/995197130 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc004490 con 0x7fd7d01031c0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7c6ffd640 1 -- 172.21.15.96:0/995197130 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc035070 con 0x7fd7d01031c0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d019e730 con 0x7fd7d01031c0 2024-04-25T17:13:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.764+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d019ec70 con 0x7fd7d01031c0 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.766+0000 7fd7c6ffd640 1 -- 172.21.15.96:0/995197130 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fd7cc042030 con 0x7fd7d01031c0 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.767+0000 7fd7c6ffd640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd7bc0afcd0 0x7fd7bc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.767+0000 7fd7d554f640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd7bc0afcd0 0x7fd7bc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.767+0000 7fd7c6ffd640 1 -- 172.21.15.96:0/995197130 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fd7cc0fef00 con 0x7fd7d01031c0 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.767+0000 7fd7dee62740 1 Processor -- start 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7d554f640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd7bc0afcd0 0x7fd7bc0b21e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7d010e720 tx=0x7fd7c0005f90 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7dee62740 1 -- start start 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39a96940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396ce2f0 0x558a39a96e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39a973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7dee62740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a399fd460 con 0x558a396ce2f0 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7dee62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558a399b7590 con 0x558a396f71b0 2024-04-25T17:13:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7dee62740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558a399f9420 con 0x558a396d64d0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7d554f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396ce2f0 0x558a39a96e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7d554f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396ce2f0 0x558a39a96e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41690/0 (socket says 172.21.15.96:41690) 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7d554f640 1 -- 172.21.15.96:0/2816120462 learned_addr learned my addr 172.21.15.96:0/2816120462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39a96940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.768+0000 7fd7d6551640 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39a973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7d5d50640 1 -- 172.21.15.96:0/2816120462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 msgr2=0x558a39a973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39a973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7d5d50640 1 -- 172.21.15.96:0/2816120462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396ce2f0 msgr2=0x558a39a96e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396ce2f0 0x558a39a96e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7d5d50640 1 -- 172.21.15.96:0/2816120462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a399c40d0 con 0x558a396f71b0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39a96940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0fc660 tx=0x7fd7cc0c68d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38fe0622f6926ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7bbfff640 1 -- 172.21.15.96:0/2816120462 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc049040 con 0x558a396f71b0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7bbfff640 1 -- 172.21.15.96:0/2816120462 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc0cb460 con 0x558a396f71b0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.769+0000 7fd7bbfff640 1 -- 172.21.15.96:0/2816120462 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0c73c0 con 0x558a396f71b0 2024-04-25T17:13:22.772 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 -- 172.21.15.96:0/2816120462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 msgr2=0x558a39a96940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39a96940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0fc660 tx=0x7fd7cc0c68d0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 -- 172.21.15.96:0/2816120462 shutdown_connections 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39a973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396ce2f0 0x558a39a96e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 --2- 172.21.15.96:0/2816120462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39a96940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 -- 172.21.15.96:0/2816120462 >> 172.21.15.96:0/2816120462 conn(0x558a396eb300 msgr2=0x558a39a932a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 -- 172.21.15.96:0/2816120462 shutdown_connections 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 -- 172.21.15.96:0/2816120462 wait complete. 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 -- start start 2024-04-25T17:13:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.770+0000 7fd7dee62740 1 Processor -- start 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39aa4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396d64d0 0x558a39aa5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7dee62740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a39aa5c50 con 0x558a396f71b0 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7dee62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558a39984b60 con 0x558a396d64d0 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7dee62740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558a39aa5dd0 con 0x558a396ce2f0 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7dee62740 10 client.0 _ll_register_callbacks cb 0x558a399a92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d5d50640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39aa4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d554f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396d64d0 0x558a39aa5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41692/0 (socket says 172.21.15.96:41692) 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 -- 172.21.15.96:0/4196587390 learned_addr learned my addr 172.21.15.96:0/4196587390 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 -- 172.21.15.96:0/4196587390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 msgr2=0x558a39aa4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39aa4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 -- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396d64d0 msgr2=0x558a39aa5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396d64d0 0x558a39aa5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d6551640 1 -- 172.21.15.96:0/4196587390 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a3970f210 con 0x558a396f71b0 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.771+0000 7fd7d554f640 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396d64d0 0x558a39aa5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7d6551640 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa3200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8011d10 tx=0x7fd7c800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af765376451130b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7b97fa640 1 -- 172.21.15.96:0/4196587390 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800fe60 con 0x558a396f71b0 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7b97fa640 1 -- 172.21.15.96:0/4196587390 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fd7c8004b20 con 0x558a396f71b0 2024-04-25T17:13:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc0cb460 con 0x558a396f71b0 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7b97fa640 1 -- 172.21.15.96:0/4196587390 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd7c8004d60 con 0x558a396f71b0 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 10 client.15603 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a397db620 con 0x558a396f71b0 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7b97fa640 1 -- 172.21.15.96:0/4196587390 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc0cb460 con 0x558a396f71b0 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7b97fa640 1 -- 172.21.15.96:0/4196587390 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd7c8019030 con 0x558a396f71b0 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 10 client.15603 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 20 client.15603 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a396b5d80 0x558a39ab5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 4 client.15603 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.772+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a39a0b070 con 0x558a396b5d80 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.773+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a396b5d80 0x558a39ab5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.773+0000 7fd7b97fa640 10 client.15603 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.773+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a396b5d80 0x558a39ab5750 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90212c48a36c543 server_cookie=18c523640a1a8d9 in_seq=0 out_seq=0 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7b97fa640 1 -- 172.21.15.96:0/4196587390 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558a39a0b070 con 0x558a396b5d80 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7b97fa640 10 client.15603 handle_command_reply: tid=0 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 client.15603 shutdown 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 msgr2=0x558a39aa3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa3200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8011d10 tx=0x7fd7c800f790 comp rx=0 tx=0).stop 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 shutdown_connections 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a396b5d80 0x558a39ab5750 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa3200 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396d64d0 0x558a39aa5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7b97fa640 0 client.15603 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 --2- 172.21.15.96:0/4196587390 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39aa4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 >> 172.21.15.96:0/4196587390 conn(0x558a396eb300 msgr2=0x558a39a972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 shutdown_connections 2024-04-25T17:13:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.774+0000 7fd7dee62740 1 -- 172.21.15.96:0/4196587390 wait complete. 2024-04-25T17:13:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.775+0000 7fd7dee62740 20 client.15603 trim_cache size 0 max 16384 2024-04-25T17:13:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 Processor -- start 2024-04-25T17:13:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 -- start start 2024-04-25T17:13:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39a9cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396d64d0 0x558a39a9d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39aa1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01ab810 con 0x558a396d64d0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01ae090 con 0x558a396f71b0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7dee62740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01abe80 con 0x558a396ce2f0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d554f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396d64d0 0x558a39a9d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d6551640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39aa1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d554f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396d64d0 0x558a39a9d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41702/0 (socket says 172.21.15.96:41702) 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d554f640 1 -- 172.21.15.96:0/3153807316 learned_addr learned my addr 172.21.15.96:0/3153807316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39a9cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d6551640 1 -- 172.21.15.96:0/3153807316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 msgr2=0x558a39a9cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d6551640 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39a9cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d6551640 1 -- 172.21.15.96:0/3153807316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396d64d0 msgr2=0x558a39a9d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d6551640 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396d64d0 0x558a39a9d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.790+0000 7fd7d6551640 1 -- 172.21.15.96:0/3153807316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a39a99740 con 0x558a396f71b0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.791+0000 7fd7d6551640 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39aa1ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80048e0 tx=0x7fd7c800fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cbbeadf17bf37b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:22.792 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.791+0000 7fd7bb7fe640 1 -- 172.21.15.96:0/3153807316 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8030070 con 0x558a396f71b0 2024-04-25T17:13:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.791+0000 7fd7bb7fe640 1 -- 172.21.15.96:0/3153807316 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c800fe60 con 0x558a396f71b0 2024-04-25T17:13:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.791+0000 7fd7bb7fe640 1 -- 172.21.15.96:0/3153807316 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c802b6a0 con 0x558a396f71b0 2024-04-25T17:13:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 -- 172.21.15.96:0/3153807316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 msgr2=0x558a39aa1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39aa1ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80048e0 tx=0x7fd7c800fcd0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 -- 172.21.15.96:0/3153807316 shutdown_connections 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396f71b0 0x558a39aa1ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396d64d0 0x558a39a9d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3153807316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396ce2f0 0x558a39a9cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 -- 172.21.15.96:0/3153807316 >> 172.21.15.96:0/3153807316 conn(0x558a396eb300 msgr2=0x558a39a91820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 -- 172.21.15.96:0/3153807316 shutdown_connections 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 -- 172.21.15.96:0/3153807316 wait complete. 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 -- start start 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.792+0000 7fd7dee62740 1 Processor -- start 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396ce2f0 0x558a39b15ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39b16430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7dee62740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7dee62740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01ac550 con 0x558a396f71b0 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7dee62740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01ad9c0 con 0x558a396ce2f0 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7dee62740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d01acc20 con 0x558a396d64d0 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7dee62740 10 client.0 _ll_register_callbacks cb 0x7fd7c8009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d554f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39b16430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d6551640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396ce2f0 0x558a39b15ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d554f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39b16430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:45566/0 (socket says 172.21.15.96:45566) 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d6551640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41704/0 (socket says 172.21.15.96:41704) 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d554f640 1 -- 172.21.15.96:0/3148954616 learned_addr learned my addr 172.21.15.96:0/3148954616 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396ce2f0 0x558a39b15ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:52768/0 (socket says 172.21.15.96:52768) 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 -- 172.21.15.96:0/3148954616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 msgr2=0x558a39b16430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39b16430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 -- 172.21.15.96:0/3148954616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 msgr2=0x558a39aa62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 -- 172.21.15.96:0/3148954616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a39aa6a00 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d6551640 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7d5d50640 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396ce2f0 0x558a39b15ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc031440 tx=0x7fd7cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d748b9f5f232106d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.793+0000 7fd7b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7b97fa640 1 -- 172.21.15.96:0/3148954616 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc049040 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7b97fa640 1 -- 172.21.15.96:0/3148954616 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fd7cc0fc8e0 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c800fe60 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7b97fa640 1 -- 172.21.15.96:0/3148954616 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0c6c70 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 10 client.25217 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a39b1d4e0 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7b97fa640 1 -- 172.21.15.96:0/3148954616 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c800fe60 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7b97fa640 1 -- 172.21.15.96:0/3148954616 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd7cc04c060 con 0x558a396ce2f0 2024-04-25T17:13:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 10 client.25217 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 10 client.25217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 20 client.25217 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a3968b1c0 0x558a39b1f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 4 client.25217 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a39a04ee0 con 0x558a3968b1c0 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.794+0000 7fd7d554f640 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a3968b1c0 0x558a39b1f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.795+0000 7fd7b97fa640 10 client.25217 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.795+0000 7fd7d554f640 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a3968b1c0 0x558a39b1f9f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dd5c47263761a9 server_cookie=8d4f448b68ff8f51 in_seq=0 out_seq=0 2024-04-25T17:13:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7b97fa640 1 -- 172.21.15.96:0/3148954616 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a39a04ee0 con 0x558a3968b1c0 2024-04-25T17:13:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7b97fa640 10 client.25217 handle_command_reply: tid=0 2024-04-25T17:13:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 client.25217 shutdown 2024-04-25T17:13:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396ce2f0 msgr2=0x558a39b15ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396ce2f0 0x558a39b15ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc031440 tx=0x7fd7cc009260 comp rx=0 tx=0).stop 2024-04-25T17:13:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 shutdown_connections 2024-04-25T17:13:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x558a3968b1c0 0x558a39b1f9f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a396f71b0 0x558a39aa62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558a396d64d0 0x558a39b16430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7b97fa640 0 client.25217 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 --2- 172.21.15.96:0/3148954616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a396ce2f0 0x558a39b15ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 >> 172.21.15.96:0/3148954616 conn(0x558a396eb300 msgr2=0x558a39aa1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 shutdown_connections 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 1 -- 172.21.15.96:0/3148954616 wait complete. 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.819+0000 7fd7dee62740 20 client.25217 trim_cache size 0 max 16384 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:13:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:13:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:13:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd7bc0afcd0 msgr2=0x7fd7bc0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd7bc0afcd0 0x7fd7bc0b21e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7d010e720 tx=0x7fd7c0005f90 comp rx=0 tx=0).stop 2024-04-25T17:13:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d01031c0 msgr2=0x7fd7d0199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d01031c0 0x7fd7d0199640 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc02a930 tx=0x7fd7cc031a20 comp rx=0 tx=0).stop 2024-04-25T17:13:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 shutdown_connections 2024-04-25T17:13:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7d0108f00 0x7fd7d019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fd7bc0afcd0 0x7fd7bc0b21e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7d0108a60 0x7fd7d0199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 --2- 172.21.15.96:0/995197130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d01031c0 0x7fd7d0199640 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 >> 172.21.15.96:0/995197130 conn(0x7fd7d00781b0 msgr2=0x7fd7d0104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 shutdown_connections 2024-04-25T17:13:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:22.820+0000 7fd7d7fb4640 1 -- 172.21.15.96:0/995197130 wait complete. 2024-04-25T17:13:22.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:22 smithi096 ceph-mon[29131]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 596 op/s 2024-04-25T17:13:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:22 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:13:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:22 smithi097 ceph-mon[32602]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 596 op/s 2024-04-25T17:13:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:22 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:13:23.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:22 smithi171 ceph-mon[33189]: pgmap v97: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 596 op/s 2024-04-25T17:13:23.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:22 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:13:23.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:23 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:13:23.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:23 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:13:23.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:23 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:13:23.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:23 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:13:23.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:23 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:13:23.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:23 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:13:24.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:23 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:13:24.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:23 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:13:24.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:23 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:13:24.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:24 smithi096 ceph-mon[29131]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-04-25T17:13:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:24 smithi097 ceph-mon[32602]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-04-25T17:13:25.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:24 smithi171 ceph-mon[33189]: pgmap v98: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-04-25T17:13:25.989 INFO:tasks.workunit.client.0.smithi096.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-25T17:13:25.989 INFO:tasks.workunit.client.0.smithi096.stdout:All 50000 operations completed A-OK! 2024-04-25T17:13:25.989 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-25T17:13:25.989 INFO:tasks.workunit.client.0.smithi096.stdout:Seed set to 1 2024-04-25T17:13:26.109 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T17:13:26.110 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T17:13:26.112 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T17:13:26.112 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T17:13:26.112 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T17:13:26.115 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T17:13:26.117 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T17:13:26.118 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x4310e9 2024-04-25T17:13:26.124 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x954a45 2024-04-25T17:13:26.158 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x9bc582 2024-04-25T17:13:26.180 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x9eecbc 2024-04-25T17:13:26.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:26 smithi096 ceph-mon[29131]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-04-25T17:13:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:26 smithi097 ceph-mon[32602]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-04-25T17:13:27.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:26 smithi171 ceph-mon[33189]: pgmap v99: 97 pgs: 97 active+clean; 41 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-04-25T17:13:28.148 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x9fdd84 2024-04-25T17:13:28.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:28 smithi096 ceph-mon[29131]: pgmap v100: 97 pgs: 97 active+clean; 38 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 535 op/s 2024-04-25T17:13:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:28 smithi097 ceph-mon[32602]: pgmap v100: 97 pgs: 97 active+clean; 38 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 535 op/s 2024-04-25T17:13:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:28 smithi171 ceph-mon[33189]: pgmap v100: 97 pgs: 97 active+clean; 38 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 535 op/s 2024-04-25T17:13:30.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:30 smithi096 ceph-mon[29131]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-25T17:13:30.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:30 smithi097 ceph-mon[32602]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-25T17:13:31.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:30 smithi171 ceph-mon[33189]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 335 op/s 2024-04-25T17:13:32.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:32 smithi096 ceph-mon[29131]: pgmap v102: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.9 MiB/s wr, 669 op/s 2024-04-25T17:13:32.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:32 smithi097 ceph-mon[32602]: pgmap v102: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.9 MiB/s wr, 669 op/s 2024-04-25T17:13:33.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:32 smithi171 ceph-mon[33189]: pgmap v102: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.9 MiB/s wr, 669 op/s 2024-04-25T17:13:34.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:34 smithi096 ceph-mon[29131]: pgmap v103: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-04-25T17:13:34.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:34 smithi097 ceph-mon[32602]: pgmap v103: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-04-25T17:13:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:34 smithi171 ceph-mon[33189]: pgmap v103: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 -- 172.21.15.96:0/3446807919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc10a040 msgr2=0x7f64cc103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 --2- 172.21.15.96:0/3446807919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc10a040 0x7f64cc103c80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009900 tx=0x7f64c002a410 comp rx=0 tx=0).stop 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 -- 172.21.15.96:0/3446807919 shutdown_connections 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 --2- 172.21.15.96:0/3446807919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64cc1042f0 0x7f64cc1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 --2- 172.21.15.96:0/3446807919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc10a040 0x7f64cc103c80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 --2- 172.21.15.96:0/3446807919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64cc109ba0 0x7f64cc103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 -- 172.21.15.96:0/3446807919 >> 172.21.15.96:0/3446807919 conn(0x7f64cc0792a0 msgr2=0x7f64cc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 -- 172.21.15.96:0/3446807919 shutdown_connections 2024-04-25T17:13:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.926+0000 7f64d1451640 1 -- 172.21.15.96:0/3446807919 wait complete. 2024-04-25T17:13:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.927+0000 7f64d1451640 1 Processor -- start 2024-04-25T17:13:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.927+0000 7f64d1451640 1 -- start start 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64d1451640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64cc1042f0 0x7f64cc19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64d1451640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc109ba0 0x7f64cc19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64d1451640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64cc10a040 0x7f64cc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64d1451640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc113de0 con 0x7f64cc109ba0 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64d1451640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc113c60 con 0x7f64cc10a040 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64d1451640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc113f60 con 0x7f64cc1042f0 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc109ba0 0x7f64cc19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64cb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64cc10a040 0x7f64cc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc109ba0 0x7f64cc19ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38254/0 (socket says 172.21.15.96:38254) 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64ca7fc640 1 -- 172.21.15.96:0/357872311 learned_addr learned my addr 172.21.15.96:0/357872311 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64caffd640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64cc1042f0 0x7f64cc19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64cb7fe640 1 -- 172.21.15.96:0/357872311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64cc1042f0 msgr2=0x7f64cc19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64cb7fe640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64cc1042f0 0x7f64cc19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64cb7fe640 1 -- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc109ba0 msgr2=0x7f64cc19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64cb7fe640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc109ba0 0x7f64cc19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64cb7fe640 1 -- 172.21.15.96:0/357872311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc19f600 con 0x7f64cc10a040 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc109ba0 0x7f64cc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.928+0000 7f64cb7fe640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64cc10a040 0x7f64cc19efc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009fd0 tx=0x7f64c400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.929+0000 7f64b3fff640 1 -- 172.21.15.96:0/357872311 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c400ce20 con 0x7f64cc10a040 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.929+0000 7f64b3fff640 1 -- 172.21.15.96:0/357872311 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c4005020 con 0x7f64cc10a040 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.929+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64cc19f8f0 con 0x7f64cc10a040 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.929+0000 7f64b3fff640 1 -- 172.21.15.96:0/357872311 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4010690 con 0x7f64cc10a040 2024-04-25T17:13:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.929+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64cc1a7270 con 0x7f64cc10a040 2024-04-25T17:13:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.931+0000 7f64b3fff640 1 -- 172.21.15.96:0/357872311 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f64c4010830 con 0x7f64cc10a040 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.931+0000 7f64d83e2740 1 Processor -- start 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.931+0000 7f64d83e2740 1 -- start start 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747251caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x55747251d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747251d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64d83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557472483460 con 0x5574721542f0 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64d83e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55747243d590 con 0x55747217d1b0 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64d83e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55747247f420 con 0x55747215c4d0 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x55747251d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64ca7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747251caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x55747251d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38258/0 (socket says 172.21.15.96:38258) 2024-04-25T17:13:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 -- 172.21.15.96:0/2349593080 learned_addr learned my addr 172.21.15.96:0/2349593080 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64b3fff640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f64b40afc50 0x7f64b40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747251d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 -- 172.21.15.96:0/2349593080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 msgr2=0x55747251d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747251d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 -- 172.21.15.96:0/2349593080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 msgr2=0x55747251caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747251caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 -- 172.21.15.96:0/2349593080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747244a0d0 con 0x5574721542f0 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64cb7fe640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f64b40afc50 0x7f64b40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64b3fff640 1 -- 172.21.15.96:0/357872311 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f64c40df6e0 con 0x7f64cc10a040 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64caffd640 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x55747251d030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009bf0 tx=0x7f64bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896167890e256110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.932+0000 7f64b17fa640 1 -- 172.21.15.96:0/2349593080 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc009e60 con 0x5574721542f0 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.933+0000 7f64b17fa640 1 -- 172.21.15.96:0/2349593080 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc004220 con 0x5574721542f0 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.933+0000 7f64b17fa640 1 -- 172.21.15.96:0/2349593080 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc019690 con 0x5574721542f0 2024-04-25T17:13:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.933+0000 7f64cb7fe640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f64b40afc50 0x7f64b40b2160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64c40dd440 tx=0x7f64c4090040 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 -- 172.21.15.96:0/2349593080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 msgr2=0x55747251d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x55747251d030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009bf0 tx=0x7f64bc011810 comp rx=0 tx=0).stop 2024-04-25T17:13:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 -- 172.21.15.96:0/2349593080 shutdown_connections 2024-04-25T17:13:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747251d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x55747251d030 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 --2- 172.21.15.96:0/2349593080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747251caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 -- 172.21.15.96:0/2349593080 >> 172.21.15.96:0/2349593080 conn(0x557472171300 msgr2=0x5574725192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 -- 172.21.15.96:0/2349593080 shutdown_connections 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 -- 172.21.15.96:0/2349593080 wait complete. 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 -- start start 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.934+0000 7f64d83e2740 1 Processor -- start 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5574721542f0 0x557472527180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x5574725276c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747217d1b0 0x55747252d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64d83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557472529c20 con 0x55747215c4d0 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64d83e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55747240ab60 con 0x5574721542f0 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64d83e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557472529da0 con 0x55747217d1b0 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64d83e2740 10 client.0 _ll_register_callbacks cb 0x55747242f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x5574725276c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x5574725276c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38266/0 (socket says 172.21.15.96:38266) 2024-04-25T17:13:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 -- 172.21.15.96:0/3665730004 learned_addr learned my addr 172.21.15.96:0/3665730004 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747217d1b0 0x55747252d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5574721542f0 0x557472527180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 -- 172.21.15.96:0/3665730004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747217d1b0 msgr2=0x55747252d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747217d1b0 0x55747252d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 -- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5574721542f0 msgr2=0x557472527180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5574721542f0 0x557472527180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.935+0000 7f64caffd640 1 -- 172.21.15.96:0/3665730004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557472195210 con 0x55747215c4d0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64caffd640 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x5574725276c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f64bc003fd0 tx=0x7f64bc009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa634c8d30fd7df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64aeff5640 1 -- 172.21.15.96:0/3665730004 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc028070 con 0x55747215c4d0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64aeff5640 1 -- 172.21.15.96:0/3665730004 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f64bc004020 con 0x55747215c4d0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64bc004220 con 0x55747215c4d0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64aeff5640 1 -- 172.21.15.96:0/3665730004 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f64bc034050 con 0x55747215c4d0 2024-04-25T17:13:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64d83e2740 10 client.15621 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557472261620 con 0x55747215c4d0 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64aeff5640 1 -- 172.21.15.96:0/3665730004 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc004220 con 0x55747215c4d0 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.936+0000 7f64aeff5640 1 -- 172.21.15.96:0/3665730004 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f64bc004a90 con 0x55747215c4d0 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64d83e2740 10 client.15621 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64d83e2740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64d83e2740 20 client.15621 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55747213bd80 0x5574725395e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64d83e2740 4 client.15621 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557472491070 con 0x55747213bd80 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55747213bd80 0x5574725395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55747213bd80 0x5574725395e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220c8ff1c54ad688 server_cookie=57406ac62f43d0fa in_seq=0 out_seq=0 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.937+0000 7f64aeff5640 10 client.15621 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.938+0000 7f64aeff5640 1 -- 172.21.15.96:0/3665730004 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557472491070 con 0x55747213bd80 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.938+0000 7f64aeff5640 10 client.15621 handle_command_reply: tid=0 2024-04-25T17:13:35.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.938+0000 7f64d83e2740 1 client.15621 shutdown 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.938+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 msgr2=0x5574725276c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.938+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x5574725276c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f64bc003fd0 tx=0x7f64bc009330 comp rx=0 tx=0).stop 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 shutdown_connections 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55747213bd80 0x5574725395e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747217d1b0 0x55747252d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x5574725276c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3665730004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5574721542f0 0x557472527180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 >> 172.21.15.96:0/3665730004 conn(0x557472171300 msgr2=0x55747251e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64aeff5640 0 client.15621 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 shutdown_connections 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 1 -- 172.21.15.96:0/3665730004 wait complete. 2024-04-25T17:13:35.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.939+0000 7f64d83e2740 20 client.15621 trim_cache size 0 max 16384 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 Processor -- start 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 -- start start 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5574721542f0 0x55747251eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x55747252d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747253ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1ada00 con 0x55747215c4d0 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1b0280 con 0x55747217d1b0 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.957+0000 7f64d83e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1ae070 con 0x5574721542f0 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x55747252d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5574721542f0 0x55747251eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x55747252d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38276/0 (socket says 172.21.15.96:38276) 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64caffd640 1 -- 172.21.15.96:0/3618846193 learned_addr learned my addr 172.21.15.96:0/3618846193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747253ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 -- 172.21.15.96:0/3618846193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 msgr2=0x55747253ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747253ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 -- 172.21.15.96:0/3618846193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 msgr2=0x55747252d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x55747252d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 -- 172.21.15.96:0/3618846193 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557472529b00 con 0x5574721542f0 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64caffd640 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x55747252d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5574721542f0 0x55747251eeb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005ab0 tx=0x7f64c0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2907d318dd497d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64b0ff9640 1 -- 172.21.15.96:0/3618846193 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0035040 con 0x5574721542f0 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64b0ff9640 1 -- 172.21.15.96:0/3618846193 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c00054d0 con 0x5574721542f0 2024-04-25T17:13:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.958+0000 7f64b0ff9640 1 -- 172.21.15.96:0/3618846193 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0047040 con 0x5574721542f0 2024-04-25T17:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 -- 172.21.15.96:0/3618846193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5574721542f0 msgr2=0x55747251eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5574721542f0 0x55747251eeb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005ab0 tx=0x7f64c0009260 comp rx=0 tx=0).stop 2024-04-25T17:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 -- 172.21.15.96:0/3618846193 shutdown_connections 2024-04-25T17:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747253ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55747215c4d0 0x55747252d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 --2- 172.21.15.96:0/3618846193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5574721542f0 0x55747251eeb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 -- 172.21.15.96:0/3618846193 >> 172.21.15.96:0/3618846193 conn(0x557472171300 msgr2=0x55747251e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 -- 172.21.15.96:0/3618846193 shutdown_connections 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.959+0000 7f64d83e2740 1 -- 172.21.15.96:0/3618846193 wait complete. 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 -- start start 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 Processor -- start 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x5574725337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747252eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747252f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1ae740 con 0x5574721542f0 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1afbb0 con 0x55747217d1b0 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc1aee10 con 0x55747215c4d0 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64d83e2740 10 client.0 _ll_register_callbacks cb 0x7f64c40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.960+0000 7f64ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x5574725337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747252eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x5574725337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38280/0 (socket says 172.21.15.96:38280) 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64ca7fc640 1 -- 172.21.15.96:0/744741193 learned_addr learned my addr 172.21.15.96:0/744741193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747252eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:38418/0 (socket says 172.21.15.96:38418) 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747252f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 -- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 msgr2=0x55747252f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747252f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 -- 172.21.15.96:0/744741193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 msgr2=0x5574725337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x5574725337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 -- 172.21.15.96:0/744741193 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747252f5a0 con 0x55747215c4d0 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x5574725337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64caffd640 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747252eb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f64bc011850 tx=0x7f64bc0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23377fcf9476a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64aeff5640 1 -- 172.21.15.96:0/744741193 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc03d060 con 0x55747215c4d0 2024-04-25T17:13:35.966 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64aeff5640 1 -- 172.21.15.96:0/744741193 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f64bc0041b0 con 0x55747215c4d0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c00054d0 con 0x55747215c4d0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64aeff5640 1 -- 172.21.15.96:0/744741193 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f64bc028070 con 0x55747215c4d0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64d83e2740 10 client.25201 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.961+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5574725a1350 con 0x55747215c4d0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64aeff5640 1 -- 172.21.15.96:0/744741193 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c00054d0 con 0x55747215c4d0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64aeff5640 1 -- 172.21.15.96:0/744741193 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f64bc004770 con 0x55747215c4d0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64d83e2740 10 client.25201 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64d83e2740 10 client.25201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64d83e2740 20 client.25201 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64d83e2740 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5574721111c0 0x5574725a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64d83e2740 4 client.25201 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55747248aee0 con 0x5574721111c0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5574721111c0 0x5574725a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64aeff5640 10 client.25201 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.962+0000 7f64ca7fc640 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5574721111c0 0x5574725a3ab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65b3b9dc5e49273 server_cookie=449a3df4424b117f in_seq=0 out_seq=0 2024-04-25T17:13:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.981+0000 7f64aeff5640 1 -- 172.21.15.96:0/744741193 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55747248aee0 con 0x5574721111c0 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.981+0000 7f64aeff5640 10 client.25201 handle_command_reply: tid=0 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.981+0000 7f64d83e2740 1 client.25201 shutdown 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.981+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 msgr2=0x55747252eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.981+0000 7f64d83e2740 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747252eb20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f64bc011850 tx=0x7f64bc0098c0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 shutdown_connections 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5574721111c0 0x5574725a3ab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55747217d1b0 0x55747252f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64aeff5640 0 client.25201 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55747215c4d0 0x55747252eb20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 --2- 172.21.15.96:0/744741193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574721542f0 0x5574725337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 >> 172.21.15.96:0/744741193 conn(0x557472171300 msgr2=0x55747252c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 shutdown_connections 2024-04-25T17:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 1 -- 172.21.15.96:0/744741193 wait complete. 2024-04-25T17:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d83e2740 20 client.25201 trim_cache size 0 max 16384 2024-04-25T17:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f64b40afc50 msgr2=0x7f64b40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f64b40afc50 0x7f64b40b2160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64c40dd440 tx=0x7f64c4090040 comp rx=0 tx=0).stop 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64cc10a040 msgr2=0x7f64cc19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64cc10a040 0x7f64cc19efc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009fd0 tx=0x7f64c400ef10 comp rx=0 tx=0).stop 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 shutdown_connections 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f64b40afc50 0x7f64b40b2160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64cc10a040 0x7f64cc19efc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64cc109ba0 0x7f64cc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 --2- 172.21.15.96:0/357872311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64cc1042f0 0x7f64cc19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.982+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 >> 172.21.15.96:0/357872311 conn(0x7f64cc0792a0 msgr2=0x7f64cc1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.983+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 shutdown_connections 2024-04-25T17:13:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:35.983+0000 7f64d1451640 1 -- 172.21.15.96:0/357872311 wait complete. 2024-04-25T17:13:36.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:36 smithi096 ceph-mon[29131]: pgmap v104: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-04-25T17:13:36.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:36 smithi097 ceph-mon[32602]: pgmap v104: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-04-25T17:13:37.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:36 smithi171 ceph-mon[33189]: pgmap v104: 97 pgs: 97 active+clean; 44 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.8 MiB/s wr, 496 op/s 2024-04-25T17:13:38.851 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x9ff773 2024-04-25T17:13:38.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:38 smithi096 ceph-mon[29131]: pgmap v105: 97 pgs: 97 active+clean; 51 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 832 op/s 2024-04-25T17:13:38.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:38 smithi097 ceph-mon[32602]: pgmap v105: 97 pgs: 97 active+clean; 51 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 832 op/s 2024-04-25T17:13:39.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:38 smithi171 ceph-mon[33189]: pgmap v105: 97 pgs: 97 active+clean; 51 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.8 MiB/s wr, 832 op/s 2024-04-25T17:13:39.248 INFO:tasks.workunit.client.0.smithi096.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-25T17:13:40.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:40 smithi096 ceph-mon[29131]: pgmap v106: 97 pgs: 97 active+clean; 51 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 669 op/s 2024-04-25T17:13:40.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:40 smithi097 ceph-mon[32602]: pgmap v106: 97 pgs: 97 active+clean; 51 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 669 op/s 2024-04-25T17:13:41.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:40 smithi171 ceph-mon[33189]: pgmap v106: 97 pgs: 97 active+clean; 51 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.8 MiB/s wr, 669 op/s 2024-04-25T17:13:42.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:42 smithi096 ceph-mon[29131]: pgmap v107: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 993 op/s 2024-04-25T17:13:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:42 smithi097 ceph-mon[32602]: pgmap v107: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 993 op/s 2024-04-25T17:13:43.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:42 smithi171 ceph-mon[33189]: pgmap v107: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 993 op/s 2024-04-25T17:13:44.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:44 smithi096 ceph-mon[29131]: pgmap v108: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T17:13:44.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:44 smithi097 ceph-mon[32602]: pgmap v108: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T17:13:45.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:44 smithi171 ceph-mon[33189]: pgmap v108: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T17:13:46.679 INFO:tasks.workunit.client.0.smithi096.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-25T17:13:46.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:46 smithi096 ceph-mon[29131]: pgmap v109: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T17:13:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:46 smithi097 ceph-mon[32602]: pgmap v109: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T17:13:47.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:46 smithi171 ceph-mon[33189]: pgmap v109: 97 pgs: 97 active+clean; 45 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T17:13:48.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:48 smithi096 ceph-mon[29131]: pgmap v110: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.6 MiB/s wr, 960 op/s 2024-04-25T17:13:48.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:48 smithi097 ceph-mon[32602]: pgmap v110: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.6 MiB/s wr, 960 op/s 2024-04-25T17:13:49.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:48 smithi171 ceph-mon[33189]: pgmap v110: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.6 MiB/s wr, 960 op/s 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 -- 172.21.15.96:0/2301562457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 msgr2=0x7fb47810f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 --2- 172.21.15.96:0/2301562457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 0x7fb47810f890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb470009ca0 tx=0x7fb47002a8c0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 -- 172.21.15.96:0/2301562457 shutdown_connections 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 --2- 172.21.15.96:0/2301562457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 0x7fb47810f890 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 --2- 172.21.15.96:0/2301562457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 0x7fb478103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 --2- 172.21.15.96:0/2301562457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb478107980 0x7fb478102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 -- 172.21.15.96:0/2301562457 >> 172.21.15.96:0/2301562457 conn(0x7fb4780778b0 msgr2=0x7fb478077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 -- 172.21.15.96:0/2301562457 shutdown_connections 2024-04-25T17:13:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.087+0000 7fb47d99d640 1 -- 172.21.15.96:0/2301562457 wait complete. 2024-04-25T17:13:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.088+0000 7fb47d99d640 1 Processor -- start 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.088+0000 7fb47d99d640 1 -- start start 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.088+0000 7fb47d99d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 0x7fb47819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb47d99d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb478107980 0x7fb47819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb47d99d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 0x7fb47819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb47d99d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb478112c30 con 0x7fb47810d470 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb47d99d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb478112ab0 con 0x7fb478107980 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb47d99d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb478112db0 con 0x7fb4781032f0 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb4777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 0x7fb47819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 0x7fb47819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 0x7fb47819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:56786/0 (socket says 172.21.15.96:56786) 2024-04-25T17:13:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb478107980 0x7fb47819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 -- 172.21.15.96:0/734656051 learned_addr learned my addr 172.21.15.96:0/734656051 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 -- 172.21.15.96:0/734656051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb478107980 msgr2=0x7fb47819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb478107980 0x7fb47819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 -- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 msgr2=0x7fb47819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 0x7fb47819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 -- 172.21.15.96:0/734656051 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47819f2c0 con 0x7fb4781032f0 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb4777fe640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 0x7fb47819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb46ffff640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb478107980 0x7fb47819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.089+0000 7fb476ffd640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 0x7fb47819a460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9f0 tx=0x7fb46800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.090+0000 7fb47c99b640 1 -- 172.21.15.96:0/734656051 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46800cd80 con 0x7fb4781032f0 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.090+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47819f5b0 con 0x7fb4781032f0 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.090+0000 7fb47c99b640 1 -- 172.21.15.96:0/734656051 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680108e0 con 0x7fb4781032f0 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.090+0000 7fb47c99b640 1 -- 172.21.15.96:0/734656051 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468018980 con 0x7fb4781032f0 2024-04-25T17:13:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.090+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4781a6f30 con 0x7fb4781032f0 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 Processor -- start 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 -- start start 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb26af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb27030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 0x55efddb27570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55efdda8d460 con 0x55efdd7871b0 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55efdda47590 con 0x55efdd75e2f0 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb48484f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55efdda89420 con 0x55efdd7664d0 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb26af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb26af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35648/0 (socket says 172.21.15.96:35648) 2024-04-25T17:13:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb47c99b640 1 -- 172.21.15.96:0/734656051 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb468010430 con 0x7fb4781032f0 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 -- 172.21.15.96:0/4138759617 learned_addr learned my addr 172.21.15.96:0/4138759617 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb476ffd640 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb27030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 -- 172.21.15.96:0/4138759617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 msgr2=0x55efddb27570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 0x55efddb27570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 -- 172.21.15.96:0/4138759617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 msgr2=0x55efddb27030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb27030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb4777fe640 1 -- 172.21.15.96:0/4138759617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efdda540d0 con 0x55efdd7871b0 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.093+0000 7fb476ffd640 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb27030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.094+0000 7fb4777fe640 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb26af0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb470009ca0 tx=0x7fb470009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fe3b32b3cafd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.094+0000 7fb47c99b640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb4640afc50 0x7fb4640b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.094+0000 7fb46dffb640 1 -- 172.21.15.96:0/4138759617 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470031650 con 0x55efdd7871b0 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.094+0000 7fb46dffb640 1 -- 172.21.15.96:0/4138759617 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb470036040 con 0x55efdd7871b0 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.094+0000 7fb46dffb640 1 -- 172.21.15.96:0/4138759617 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4700044a0 con 0x55efdd7871b0 2024-04-25T17:13:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.094+0000 7fb476ffd640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb4640afc50 0x7fb4640b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.094+0000 7fb47c99b640 1 -- 172.21.15.96:0/734656051 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fb4680df340 con 0x7fb4781032f0 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 -- 172.21.15.96:0/4138759617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 msgr2=0x55efddb26af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb26af0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb470009ca0 tx=0x7fb470009980 comp rx=0 tx=0).stop 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb476ffd640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb4640afc50 0x7fb4640b2160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4680dcb50 tx=0x7fb468000bc0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 -- 172.21.15.96:0/4138759617 shutdown_connections 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 0x55efddb27570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb27030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 --2- 172.21.15.96:0/4138759617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb26af0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 -- 172.21.15.96:0/4138759617 >> 172.21.15.96:0/4138759617 conn(0x55efdd77b300 msgr2=0x55efddb232a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 -- 172.21.15.96:0/4138759617 shutdown_connections 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 -- 172.21.15.96:0/4138759617 wait complete. 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.095+0000 7fb48484f740 1 -- start start 2024-04-25T17:13:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 1 Processor -- start 2024-04-25T17:13:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb32b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb33050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7871b0 0x55efddb30fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55efddb33bc0 con 0x55efdd7664d0 2024-04-25T17:13:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55efdda14b60 con 0x55efdd75e2f0 2024-04-25T17:13:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55efddb33d40 con 0x55efdd7871b0 2024-04-25T17:13:49.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb33050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb48484f740 10 client.0 _ll_register_callbacks cb 0x55efdda392b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb33050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35650/0 (socket says 172.21.15.96:35650) 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb46ffff640 1 -- 172.21.15.96:0/3488074695 learned_addr learned my addr 172.21.15.96:0/3488074695 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb32b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.096+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7871b0 0x55efddb30fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb46ffff640 1 -- 172.21.15.96:0/3488074695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7871b0 msgr2=0x55efddb30fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb46ffff640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7871b0 0x55efddb30fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb46ffff640 1 -- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 msgr2=0x55efddb32b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb46ffff640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb32b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb46ffff640 1 -- 172.21.15.96:0/3488074695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efdd79f210 con 0x55efdd7664d0 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb32b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb46ffff640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb33050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb460011970 tx=0x7fb460011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3fc4f9ac3bf16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb456ffd640 1 -- 172.21.15.96:0/3488074695 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000fb70 con 0x55efdd7664d0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb456ffd640 1 -- 172.21.15.96:0/3488074695 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fb46000fd10 con 0x55efdd7664d0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb470036040 con 0x55efdd7664d0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb456ffd640 1 -- 172.21.15.96:0/3488074695 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb460004020 con 0x55efdd7664d0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb48484f740 10 client.15639 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55efdd86b620 con 0x55efdd7664d0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.097+0000 7fb456ffd640 1 -- 172.21.15.96:0/3488074695 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb470036040 con 0x55efdd7664d0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb456ffd640 1 -- 172.21.15.96:0/3488074695 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb460019410 con 0x55efdd7664d0 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb48484f740 10 client.15639 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb48484f740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb48484f740 20 client.15639 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb48484f740 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd745d80 0x55efddb436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb48484f740 4 client.15639 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:13:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efdda9b070 con 0x55efdd745d80 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd745d80 0x55efddb436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd745d80 0x55efddb436f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a143f6e6677fd6 server_cookie=f140e7357316ff91 in_seq=0 out_seq=0 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.098+0000 7fb456ffd640 10 client.15639 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb456ffd640 1 -- 172.21.15.96:0/3488074695 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55efdda9b070 con 0x55efdd745d80 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb456ffd640 10 client.15639 handle_command_reply: tid=0 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 client.15639 shutdown 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 msgr2=0x55efddb33050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb33050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb460011970 tx=0x7fb460011d70 comp rx=0 tx=0).stop 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 shutdown_connections 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd745d80 0x55efddb436f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7871b0 0x55efddb30fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb33050 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 --2- 172.21.15.96:0/3488074695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb32b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 >> 172.21.15.96:0/3488074695 conn(0x55efdd77b300 msgr2=0x55efddb26920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb456ffd640 0 client.15639 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 shutdown_connections 2024-04-25T17:13:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 1 -- 172.21.15.96:0/3488074695 wait complete. 2024-04-25T17:13:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.099+0000 7fb48484f740 20 client.15639 trim_cache size 0 max 16384 2024-04-25T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.116+0000 7fb48484f740 1 Processor -- start 2024-04-25T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb48484f740 1 -- start start 2024-04-25T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb48484f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb48ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb48484f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 0x55efddb49430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb48484f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb38fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb48484f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ad510 con 0x55efdd7871b0 2024-04-25T17:13:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb48484f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781afd90 con 0x55efdd75e2f0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb48484f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781adb80 con 0x55efdd7664d0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb38fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb38fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35664/0 (socket says 172.21.15.96:35664) 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 -- 172.21.15.96:0/3197605931 learned_addr learned my addr 172.21.15.96:0/3197605931 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb48ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb46ffff640 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 0x55efddb49430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 -- 172.21.15.96:0/3197605931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 msgr2=0x55efddb49430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 0x55efddb49430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 -- 172.21.15.96:0/3197605931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 msgr2=0x55efddb48ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb48ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.117+0000 7fb4777fe640 1 -- 172.21.15.96:0/3197605931 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efddb278a0 con 0x55efdd7871b0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.118+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb38fd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb470004150 tx=0x7fb4700040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fa4f0512cdcf58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.118+0000 7fb46d7fa640 1 -- 172.21.15.96:0/3197605931 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470031760 con 0x55efdd7871b0 2024-04-25T17:13:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.118+0000 7fb46d7fa640 1 -- 172.21.15.96:0/3197605931 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb470036030 con 0x55efdd7871b0 2024-04-25T17:13:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.118+0000 7fb46d7fa640 1 -- 172.21.15.96:0/3197605931 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47002fb00 con 0x55efdd7871b0 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 -- 172.21.15.96:0/3197605931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 msgr2=0x55efddb38fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb38fd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb470004150 tx=0x7fb4700040e0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 -- 172.21.15.96:0/3197605931 shutdown_connections 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7871b0 0x55efddb38fd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd7664d0 0x55efddb49430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 --2- 172.21.15.96:0/3197605931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd75e2f0 0x55efddb48ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 -- 172.21.15.96:0/3197605931 >> 172.21.15.96:0/3197605931 conn(0x55efdd77b300 msgr2=0x55efddb2f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 -- 172.21.15.96:0/3197605931 shutdown_connections 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 -- 172.21.15.96:0/3197605931 wait complete. 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 -- start start 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.119+0000 7fb48484f740 1 Processor -- start 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb48484f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd75e2f0 0x55efddb382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb48484f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb48484f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd7871b0 0x55efddbadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb48484f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ae250 con 0x55efdd7664d0 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb48484f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781af6c0 con 0x55efdd7871b0 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb48484f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781ae920 con 0x55efdd75e2f0 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb48484f740 10 client.0 _ll_register_callbacks cb 0x7fb460009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35680/0 (socket says 172.21.15.96:35680) 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 -- 172.21.15.96:0/3032084475 learned_addr learned my addr 172.21.15.96:0/3032084475 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:13:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd7871b0 0x55efddbadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd75e2f0 0x55efddb382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 -- 172.21.15.96:0/3032084475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd75e2f0 msgr2=0x55efddb382a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd75e2f0 0x55efddb382a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 -- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd7871b0 msgr2=0x55efddbadec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd7871b0 0x55efddbadec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 -- 172.21.15.96:0/3032084475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efddbae400 con 0x55efdd7664d0 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb46ffff640 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb387e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb460001eb0 tx=0x7fb46000fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f252b0965c5a507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.120+0000 7fb456ffd640 1 -- 172.21.15.96:0/3032084475 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460005230 con 0x55efdd7664d0 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb456ffd640 1 -- 172.21.15.96:0/3032084475 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fb4600053d0 con 0x55efdd7664d0 2024-04-25T17:13:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb470036030 con 0x55efdd7664d0 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb456ffd640 1 -- 172.21.15.96:0/3032084475 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb4600055f0 con 0x55efdd7664d0 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 10 client.15645 fetch_fsmap learned FSMap version 20 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55efddbaf5c0 con 0x55efdd7664d0 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb456ffd640 1 -- 172.21.15.96:0/3032084475 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb470036030 con 0x55efdd7664d0 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb456ffd640 1 -- 172.21.15.96:0/3032084475 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb460003c00 con 0x55efdd7664d0 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 10 client.15645 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 20 client.15645 populate_metadata read hostname 'smithi096' 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd71b1c0 0x55efddbb1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 4 client.15645 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efdda94ee0 con 0x55efdd71b1c0 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.121+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd71b1c0 0x55efddbb1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.122+0000 7fb456ffd640 10 client.15645 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.122+0000 7fb4777fe640 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd71b1c0 0x55efddbb1dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd487d160efd74f server_cookie=fcd782fbde56a655 in_seq=0 out_seq=0 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb456ffd640 1 -- 172.21.15.96:0/3032084475 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efdda94ee0 con 0x55efdd71b1c0 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb456ffd640 10 client.15645 handle_command_reply: tid=0 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 client.15645 shutdown 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 msgr2=0x55efddb387e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb387e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb460001eb0 tx=0x7fb46000fe20 comp rx=0 tx=0).stop 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 shutdown_connections 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55efdd71b1c0 0x55efddbb1dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55efdd7871b0 0x55efddbadec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efdd7664d0 0x55efddb387e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb456ffd640 0 client.15645 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 --2- 172.21.15.96:0/3032084475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55efdd75e2f0 0x55efddb382a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 >> 172.21.15.96:0/3032084475 conn(0x55efdd77b300 msgr2=0x55efddb2f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 shutdown_connections 2024-04-25T17:13:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 1 -- 172.21.15.96:0/3032084475 wait complete. 2024-04-25T17:13:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.243+0000 7fb48484f740 20 client.15645 trim_cache size 0 max 16384 2024-04-25T17:13:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:13:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:13:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:13:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:13:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb4640afc50 msgr2=0x7fb4640b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb4640afc50 0x7fb4640b2160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb4680dcb50 tx=0x7fb468000bc0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 msgr2=0x7fb47819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:13:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 0x7fb47819a460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9f0 tx=0x7fb46800eeb0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 shutdown_connections 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb47810d470 0x7fb47819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb4640afc50 0x7fb4640b2160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb478107980 0x7fb47819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 --2- 172.21.15.96:0/734656051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4781032f0 0x7fb47819a460 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 >> 172.21.15.96:0/734656051 conn(0x7fb4780778b0 msgr2=0x7fb478103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 shutdown_connections 2024-04-25T17:13:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:13:49.244+0000 7fb47d99d640 1 -- 172.21.15.96:0/734656051 wait complete. 2024-04-25T17:13:50.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:50 smithi096 ceph-mon[29131]: pgmap v111: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T17:13:50.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:50 smithi097 ceph-mon[32602]: pgmap v111: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T17:13:51.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:50 smithi171 ceph-mon[33189]: pgmap v111: 97 pgs: 97 active+clean; 46 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 623 op/s 2024-04-25T17:13:52.129 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x9ffb01 2024-04-25T17:13:52.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:52 smithi096 ceph-mon[29131]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.3 MiB/s wr, 867 op/s 2024-04-25T17:13:52.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:52 smithi097 ceph-mon[32602]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.3 MiB/s wr, 867 op/s 2024-04-25T17:13:53.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:52 smithi171 ceph-mon[33189]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.3 MiB/s wr, 867 op/s 2024-04-25T17:13:54.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:54 smithi096 ceph-mon[29131]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 543 op/s 2024-04-25T17:13:54.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:54 smithi097 ceph-mon[32602]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 543 op/s 2024-04-25T17:13:55.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:54 smithi171 ceph-mon[33189]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 543 op/s 2024-04-25T17:13:56.232 INFO:tasks.workunit.client.0.smithi096.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-25T17:13:56.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:56 smithi096 ceph-mon[29131]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 543 op/s 2024-04-25T17:13:56.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:56 smithi097 ceph-mon[32602]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 543 op/s 2024-04-25T17:13:57.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:56 smithi171 ceph-mon[33189]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 543 op/s 2024-04-25T17:13:58.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:13:58 smithi096 ceph-mon[29131]: pgmap v115: 97 pgs: 97 active+clean; 40 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 802 op/s 2024-04-25T17:13:58.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:13:58 smithi097 ceph-mon[32602]: pgmap v115: 97 pgs: 97 active+clean; 40 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 802 op/s 2024-04-25T17:13:59.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:13:58 smithi171 ceph-mon[33189]: pgmap v115: 97 pgs: 97 active+clean; 40 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 802 op/s 2024-04-25T17:14:00.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:00 smithi096 ceph-mon[29131]: pgmap v116: 97 pgs: 97 active+clean; 40 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.2 MiB/s wr, 502 op/s 2024-04-25T17:14:00.983 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:00 smithi097 ceph-mon[32602]: pgmap v116: 97 pgs: 97 active+clean; 40 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.2 MiB/s wr, 502 op/s 2024-04-25T17:14:01.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:00 smithi171 ceph-mon[33189]: pgmap v116: 97 pgs: 97 active+clean; 40 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.2 MiB/s wr, 502 op/s 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 -- 172.21.15.96:0/323606372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac104cd0 msgr2=0x7f66ac1050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/323606372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac104cd0 0x7f66ac1050e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f669c009900 tx=0x7f669c02a510 comp rx=0 tx=0).stop 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 -- 172.21.15.96:0/323606372 shutdown_connections 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/323606372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac10e0e0 0x7f66ac1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/323606372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac10b730 0x7f66ac10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/323606372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac104cd0 0x7f66ac1050e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 -- 172.21.15.96:0/323606372 >> 172.21.15.96:0/323606372 conn(0x7f66ac0781b0 msgr2=0x7f66ac0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 -- 172.21.15.96:0/323606372 shutdown_connections 2024-04-25T17:14:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.349+0000 7f66b0f3f640 1 -- 172.21.15.96:0/323606372 wait complete. 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.350+0000 7f66b0f3f640 1 Processor -- start 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.350+0000 7f66b0f3f640 1 -- start start 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.350+0000 7f66b0f3f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac104cd0 0x7f66ac19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.350+0000 7f66b0f3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac10b730 0x7f66ac19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.350+0000 7f66b0f3f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac10e0e0 0x7f66ac19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66b0f3f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac113d40 con 0x7f66ac10b730 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66b0f3f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac113bc0 con 0x7f66ac10e0e0 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66b0f3f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac113ec0 con 0x7f66ac104cd0 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac104cd0 0x7f66ac19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac10b730 0x7f66ac19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac104cd0 0x7f66ac19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:60814/0 (socket says 172.21.15.96:60814) 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac10b730 0x7f66ac19ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33592/0 (socket says 172.21.15.96:33592) 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 -- 172.21.15.96:0/3008519950 learned_addr learned my addr 172.21.15.96:0/3008519950 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac10e0e0 0x7f66ac19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 -- 172.21.15.96:0/3008519950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac10e0e0 msgr2=0x7f66ac19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac10e0e0 0x7f66ac19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 -- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac10b730 msgr2=0x7f66ac19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac10b730 0x7f66ac19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 -- 172.21.15.96:0/3008519950 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ac19f640 con 0x7f66ac104cd0 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac10e0e0 0x7f66ac19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac10b730 0x7f66ac19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f66aa59c640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac104cd0 0x7f66ac19a7e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f669c005940 tx=0x7f669c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.351+0000 7f669b7fe640 1 -- 172.21.15.96:0/3008519950 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c03f030 con 0x7f66ac104cd0 2024-04-25T17:14:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.352+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ac19f8d0 con 0x7f66ac104cd0 2024-04-25T17:14:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.352+0000 7f669b7fe640 1 -- 172.21.15.96:0/3008519950 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669c003aa0 con 0x7f66ac104cd0 2024-04-25T17:14:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.352+0000 7f669b7fe640 1 -- 172.21.15.96:0/3008519950 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c035070 con 0x7f66ac104cd0 2024-04-25T17:14:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.352+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ac19fdc0 con 0x7f66ac104cd0 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.354+0000 7f669b7fe640 1 -- 172.21.15.96:0/3008519950 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f669c042030 con 0x7f66ac104cd0 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.354+0000 7f66b7df4740 1 Processor -- start 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66b7df4740 1 -- start start 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da863439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86343ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86344420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66b7df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da862aa460 con 0x55da85fa41b0 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66b7df4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da86264590 con 0x55da85f7b2f0 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66b7df4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55da862a6420 con 0x55da85f834d0 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f669b7fe640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f66940afc50 0x7f66940b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66aad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da863439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86343ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66aad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da863439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33606/0 (socket says 172.21.15.96:33606) 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66aad9d640 1 -- 172.21.15.96:0/3667838621 learned_addr learned my addr 172.21.15.96:0/3667838621 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86344420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f66940afc50 0x7f66940b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f669b7fe640 1 -- 172.21.15.96:0/3008519950 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f669c0ff800 con 0x7f66ac104cd0 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66a9d9b640 1 -- 172.21.15.96:0/3667838621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 msgr2=0x55da86343ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86343ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66a9d9b640 1 -- 172.21.15.96:0/3667838621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 msgr2=0x55da863439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da863439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66a9d9b640 1 -- 172.21.15.96:0/3667838621 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da862710d0 con 0x55da85f834d0 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66aa59c640 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86343ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.355+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da863439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.356+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f66940afc50 0x7f66940b2160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f66a4005c40 tx=0x7f66a4009a80 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.356+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86344420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009fa0 tx=0x7f66a00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac1ad013784d891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.356+0000 7f6698ff9640 1 -- 172.21.15.96:0/3667838621 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0004030 con 0x55da85f834d0 2024-04-25T17:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.356+0000 7f6698ff9640 1 -- 172.21.15.96:0/3667838621 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00041d0 con 0x55da85f834d0 2024-04-25T17:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.356+0000 7f6698ff9640 1 -- 172.21.15.96:0/3667838621 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000f800 con 0x55da85f834d0 2024-04-25T17:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 -- 172.21.15.96:0/3667838621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 msgr2=0x55da86344420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86344420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009fa0 tx=0x7f66a00118d0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 -- 172.21.15.96:0/3667838621 shutdown_connections 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86344420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86343ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3667838621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da863439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 -- 172.21.15.96:0/3667838621 >> 172.21.15.96:0/3667838621 conn(0x55da85f98300 msgr2=0x55da863402a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 -- 172.21.15.96:0/3667838621 shutdown_connections 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.357+0000 7f66b7df4740 1 -- 172.21.15.96:0/3667838621 wait complete. 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 -- start start 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 Processor -- start 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da863525c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f834d0 0x55da86352b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da86358520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da86355030 con 0x55da85fa41b0 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55da86231b60 con 0x55da85f834d0 2024-04-25T17:14:02.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55da863551b0 con 0x55da85f7b2f0 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66b7df4740 10 client.0 _ll_register_callbacks cb 0x55da862562b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66aad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da86358520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da863525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da863525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:60840/0 (socket says 172.21.15.96:60840) 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66aa59c640 1 -- 172.21.15.96:0/1912132790 learned_addr learned my addr 172.21.15.96:0/1912132790 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.358+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f834d0 0x55da86352b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66aa59c640 1 -- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f834d0 msgr2=0x55da86352b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66aa59c640 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f834d0 0x55da86352b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66aa59c640 1 -- 172.21.15.96:0/1912132790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 msgr2=0x55da86358520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66aa59c640 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da86358520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66aa59c640 1 -- 172.21.15.96:0/1912132790 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da85fbc210 con 0x55da85f7b2f0 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66aad9d640 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da86358520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66aa59c640 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da863525c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c093010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f2e176bf9506109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f6691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f6691ffb640 1 -- 172.21.15.96:0/1912132790 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c0c7300 con 0x55da85f7b2f0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f6691ffb640 1 -- 172.21.15.96:0/1912132790 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f669c0c3fa0 con 0x55da85f7b2f0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a00041d0 con 0x55da85f7b2f0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f6691ffb640 1 -- 172.21.15.96:0/1912132790 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f669c049020 con 0x55da85f7b2f0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66b7df4740 10 client.25228 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55da86088620 con 0x55da85f7b2f0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.359+0000 7f6691ffb640 1 -- 172.21.15.96:0/1912132790 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a00041d0 con 0x55da85f7b2f0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f6691ffb640 1 -- 172.21.15.96:0/1912132790 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f669c0c8ce0 con 0x55da85f7b2f0 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66b7df4740 10 client.25228 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66b7df4740 10 client.25228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66b7df4740 20 client.25228 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85f62d80 0x55da86364b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66b7df4740 4 client.25228 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da862b8070 con 0x55da85f62d80 2024-04-25T17:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85f62d80 0x55da86364b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f6691ffb640 10 client.25228 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.360+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85f62d80 0x55da86364b10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c0ef66d047f339 server_cookie=53d20210b1c4a87e in_seq=0 out_seq=0 2024-04-25T17:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f6691ffb640 1 -- 172.21.15.96:0/1912132790 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da862b8070 con 0x55da85f62d80 2024-04-25T17:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f6691ffb640 10 client.25228 handle_command_reply: tid=0 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 client.25228 shutdown 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 msgr2=0x55da863525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da863525c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f669c006010 tx=0x7f669c093010 comp rx=0 tx=0).stop 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 shutdown_connections 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85f62d80 0x55da86364b10 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85fa41b0 0x55da86358520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f6691ffb640 0 client.25228 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f834d0 0x55da86352b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1912132790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da863525c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 >> 172.21.15.96:0/1912132790 conn(0x55da85f98300 msgr2=0x55da863457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 shutdown_connections 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 1 -- 172.21.15.96:0/1912132790 wait complete. 2024-04-25T17:14:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.361+0000 7f66b7df4740 20 client.25228 trim_cache size 0 max 16384 2024-04-25T17:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 Processor -- start 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 -- start start 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f62d80 0x55da863516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86358420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86343f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1ab780 con 0x55da85f62d80 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1adfe0 con 0x55da85f7b2f0 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66b7df4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1abdd0 con 0x55da85f834d0 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f62d80 0x55da863516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.378+0000 7f66aad9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86343f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f62d80 0x55da863516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33630/0 (socket says 172.21.15.96:33630) 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 -- 172.21.15.96:0/1145807042 learned_addr learned my addr 172.21.15.96:0/1145807042 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86358420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 -- 172.21.15.96:0/1145807042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 msgr2=0x55da86343f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86343f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 -- 172.21.15.96:0/1145807042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 msgr2=0x55da86358420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86358420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 -- 172.21.15.96:0/1145807042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da86354f10 con 0x55da85f62d80 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aad9d640 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86343f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f66aa59c640 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f62d80 0x55da863516b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f669c0cbe00 tx=0x7f669c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb4c37ff64e667e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f6693fff640 1 -- 172.21.15.96:0/1145807042 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c0c7120 con 0x55da85f62d80 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f6693fff640 1 -- 172.21.15.96:0/1145807042 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669c097180 con 0x55da85f62d80 2024-04-25T17:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.379+0000 7f6693fff640 1 -- 172.21.15.96:0/1145807042 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c0c79b0 con 0x55da85f62d80 2024-04-25T17:14:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 -- 172.21.15.96:0/1145807042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f62d80 msgr2=0x55da863516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f62d80 0x55da863516b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f669c0cbe00 tx=0x7f669c005c90 comp rx=0 tx=0).stop 2024-04-25T17:14:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 -- 172.21.15.96:0/1145807042 shutdown_connections 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f834d0 0x55da86343f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f7b2f0 0x55da86358420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 --2- 172.21.15.96:0/1145807042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f62d80 0x55da863516b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 -- 172.21.15.96:0/1145807042 >> 172.21.15.96:0/1145807042 conn(0x55da85f98300 msgr2=0x55da86343880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 -- 172.21.15.96:0/1145807042 shutdown_connections 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.380+0000 7f66b7df4740 1 -- 172.21.15.96:0/1145807042 wait complete. 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 -- start start 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 Processor -- start 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f62d80 0x55da86359a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da86359f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f834d0 0x55da8635a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1ac4a0 con 0x55da85f834d0 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1ad910 con 0x55da85f62d80 2024-04-25T17:14:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f66ac1acb70 con 0x55da85f7b2f0 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66b7df4740 10 client.0 _ll_register_callbacks cb 0x7f669c0c3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66aad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f834d0 0x55da8635a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66aad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f834d0 0x55da8635a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33644/0 (socket says 172.21.15.96:33644) 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66aa59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f62d80 0x55da86359a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66aad9d640 1 -- 172.21.15.96:0/3361399170 learned_addr learned my addr 172.21.15.96:0/3361399170 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.381+0000 7f66a9d9b640 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da86359f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66aad9d640 1 -- 172.21.15.96:0/3361399170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 msgr2=0x55da86359f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da86359f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66aad9d640 1 -- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f62d80 msgr2=0x55da86359a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f62d80 0x55da86359a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66aad9d640 1 -- 172.21.15.96:0/3361399170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da863cae70 con 0x55da85f834d0 2024-04-25T17:14:02.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66aa59c640 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f62d80 0x55da86359a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66aad9d640 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f834d0 0x55da8635a4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f66a4020630 tx=0x7f66a4009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11e8de7b5763478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f6691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f6691ffb640 1 -- 172.21.15.96:0/3361399170 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a4004b20 con 0x55da85f834d0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da862b1ee0 con 0x55da85f834d0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f6691ffb640 1 -- 172.21.15.96:0/3361399170 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f66a401b040 con 0x55da85f834d0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f6691ffb640 1 -- 172.21.15.96:0/3361399170 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f66a4037050 con 0x55da85f834d0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66b7df4740 10 client.15663 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55da863cc0f0 con 0x55da85f834d0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.382+0000 7f6691ffb640 1 -- 172.21.15.96:0/3361399170 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55da862b1ee0 con 0x55da85f834d0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f6691ffb640 1 -- 172.21.15.96:0/3361399170 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f66a401f070 con 0x55da85f834d0 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66b7df4740 10 client.15663 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66b7df4740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66b7df4740 20 client.15663 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85fa41b0 0x55da863ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66b7df4740 4 client.15663 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:14:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da8622d5f0 con 0x55da85fa41b0 2024-04-25T17:14:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66aa59c640 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85fa41b0 0x55da863ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f6691ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.383+0000 7f66aa59c640 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85fa41b0 0x55da863ce8e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e6874975df0712 server_cookie=e6bc24fcf956b92 in_seq=0 out_seq=0 2024-04-25T17:14:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.401+0000 7f6691ffb640 1 -- 172.21.15.96:0/3361399170 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da8622d5f0 con 0x55da85fa41b0 2024-04-25T17:14:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.401+0000 7f6691ffb640 10 client.15663 handle_command_reply: tid=0 2024-04-25T17:14:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.401+0000 7f66b7df4740 1 client.15663 shutdown 2024-04-25T17:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f834d0 msgr2=0x55da8635a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f834d0 0x55da8635a4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f66a4020630 tx=0x7f66a4009760 comp rx=0 tx=0).stop 2024-04-25T17:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 shutdown_connections 2024-04-25T17:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x55da85fa41b0 0x55da863ce8e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da85f834d0 0x55da8635a4a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55da85f7b2f0 0x55da86359f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 --2- 172.21.15.96:0/3361399170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55da85f62d80 0x55da86359a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 >> 172.21.15.96:0/3361399170 conn(0x55da85f98300 msgr2=0x55da86357410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.403+0000 7f6691ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.404+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 shutdown_connections 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.404+0000 7f66b7df4740 1 -- 172.21.15.96:0/3361399170 wait complete. 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.404+0000 7f66b7df4740 20 client.15663 trim_cache size 0 max 16384 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f66940afc50 msgr2=0x7f66940b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f66940afc50 0x7f66940b2160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f66a4005c40 tx=0x7f66a4009a80 comp rx=0 tx=0).stop 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac104cd0 msgr2=0x7f66ac19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac104cd0 0x7f66ac19a7e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f669c005940 tx=0x7f669c004270 comp rx=0 tx=0).stop 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 shutdown_connections 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f66940afc50 0x7f66940b2160 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac10e0e0 0x7f66ac19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66ac10b730 0x7f66ac19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 --2- 172.21.15.96:0/3008519950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66ac104cd0 0x7f66ac19a7e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 >> 172.21.15.96:0/3008519950 conn(0x7f66ac0781b0 msgr2=0x7f66ac101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 shutdown_connections 2024-04-25T17:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:02.405+0000 7f66b0f3f640 1 -- 172.21.15.96:0/3008519950 wait complete. 2024-04-25T17:14:02.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:02 smithi096 ceph-mon[29131]: pgmap v117: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 766 op/s 2024-04-25T17:14:02.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:02 smithi097 ceph-mon[32602]: pgmap v117: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 766 op/s 2024-04-25T17:14:03.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:02 smithi171 ceph-mon[33189]: pgmap v117: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 766 op/s 2024-04-25T17:14:04.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:04 smithi096 ceph-mon[29131]: pgmap v118: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-04-25T17:14:04.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:04 smithi097 ceph-mon[32602]: pgmap v118: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-04-25T17:14:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:04 smithi171 ceph-mon[33189]: pgmap v118: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-04-25T17:14:05.302 INFO:tasks.workunit.client.0.smithi096.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-25T17:14:05.302 INFO:tasks.workunit.client.0.smithi096.stdout:All 50000 operations completed A-OK! 2024-04-25T17:14:05.303 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-25T17:14:05.303 INFO:tasks.workunit.client.0.smithi096.stdout:Seed set to 1 2024-04-25T17:14:06.404 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T17:14:06.405 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T17:14:06.408 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T17:14:06.408 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T17:14:06.408 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T17:14:06.411 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T17:14:06.413 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T17:14:06.414 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T17:14:06.425 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x1c37b90 2024-04-25T17:14:06.432 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x5bd330d 2024-04-25T17:14:06.452 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x637bf99 2024-04-25T17:14:06.637 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x63d9b9c 2024-04-25T17:14:06.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:06 smithi096 ceph-mon[29131]: pgmap v119: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-04-25T17:14:06.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:06 smithi097 ceph-mon[32602]: pgmap v119: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-04-25T17:14:07.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:06 smithi171 ceph-mon[33189]: pgmap v119: 97 pgs: 97 active+clean; 49 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2024-04-25T17:14:08.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:08 smithi096 ceph-mon[29131]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 745 op/s 2024-04-25T17:14:08.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:08 smithi097 ceph-mon[32602]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 745 op/s 2024-04-25T17:14:09.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:08 smithi171 ceph-mon[33189]: pgmap v120: 97 pgs: 97 active+clean; 46 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 745 op/s 2024-04-25T17:14:10.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:10 smithi096 ceph-mon[29131]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 486 op/s 2024-04-25T17:14:10.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:10 smithi097 ceph-mon[32602]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 486 op/s 2024-04-25T17:14:11.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:10 smithi171 ceph-mon[33189]: pgmap v121: 97 pgs: 97 active+clean; 46 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.9 MiB/s wr, 486 op/s 2024-04-25T17:14:12.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:12 smithi096 ceph-mon[29131]: pgmap v122: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 846 op/s 2024-04-25T17:14:12.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:12 smithi097 ceph-mon[32602]: pgmap v122: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 846 op/s 2024-04-25T17:14:13.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:12 smithi171 ceph-mon[33189]: pgmap v122: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 846 op/s 2024-04-25T17:14:14.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:14 smithi096 ceph-mon[29131]: pgmap v123: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.3 MiB/s wr, 582 op/s 2024-04-25T17:14:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:14 smithi097 ceph-mon[32602]: pgmap v123: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.3 MiB/s wr, 582 op/s 2024-04-25T17:14:15.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:14 smithi171 ceph-mon[33189]: pgmap v123: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.3 MiB/s wr, 582 op/s 2024-04-25T17:14:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 -- 172.21.15.96:0/2164654061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 msgr2=0x7f7788106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 --2- 172.21.15.96:0/2164654061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788106e10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7778009900 tx=0x7f777802a410 comp rx=0 tx=0).stop 2024-04-25T17:14:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 -- 172.21.15.96:0/2164654061 shutdown_connections 2024-04-25T17:14:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 --2- 172.21.15.96:0/2164654061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77881074a0 0x7f778810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 --2- 172.21.15.96:0/2164654061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788106e10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 --2- 172.21.15.96:0/2164654061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788102450 0x7f7788102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 -- 172.21.15.96:0/2164654061 >> 172.21.15.96:0/2164654061 conn(0x7f77880778b0 msgr2=0x7f7788077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 -- 172.21.15.96:0/2164654061 shutdown_connections 2024-04-25T17:14:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.519+0000 7f778ffc5640 1 -- 172.21.15.96:0/2164654061 wait complete. 2024-04-25T17:14:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 Processor -- start 2024-04-25T17:14:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 -- start start 2024-04-25T17:14:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788102450 0x7f7788199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77881074a0 0x7f778819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7788112c40 con 0x7f77881069e0 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7788112ac0 con 0x7f77881074a0 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778ffc5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7788112dc0 con 0x7f7788102450 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41978/0 (socket says 172.21.15.96:41978) 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 -- 172.21.15.96:0/4060963659 learned_addr learned my addr 172.21.15.96:0/4060963659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778dd61640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788102450 0x7f7788199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778e562640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77881074a0 0x7f778819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 -- 172.21.15.96:0/4060963659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788102450 msgr2=0x7f7788199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788102450 0x7f7788199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 -- 172.21.15.96:0/4060963659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77881074a0 msgr2=0x7f778819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77881074a0 0x7f778819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778d560640 1 -- 172.21.15.96:0/4060963659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778819e4e0 con 0x7f77881069e0 2024-04-25T17:14:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.520+0000 7f778dd61640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788102450 0x7f7788199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.521+0000 7f778d560640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788199be0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7778006010 tx=0x7f7778004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.521+0000 7f777effd640 1 -- 172.21.15.96:0/4060963659 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777803f030 con 0x7f77881069e0 2024-04-25T17:14:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.521+0000 7f777effd640 1 -- 172.21.15.96:0/4060963659 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7778003aa0 con 0x7f77881069e0 2024-04-25T17:14:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.521+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778819e770 con 0x7f77881069e0 2024-04-25T17:14:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.521+0000 7f777effd640 1 -- 172.21.15.96:0/4060963659 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778035070 con 0x7f77881069e0 2024-04-25T17:14:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.521+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778819ec60 con 0x7f77881069e0 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.523+0000 7f777effd640 1 -- 172.21.15.96:0/4060963659 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7778042030 con 0x7f77881069e0 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 Processor -- start 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 -- start start 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0f21b0 0x5583fc491af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc492030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0d14d0 0x5583fc492570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583fc3f8460 con 0x5583fc0f21b0 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583fc3b2590 con 0x5583fc0d14d0 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f7796e79740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5583fc3f4420 con 0x5583fc0c92f0 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f778dd61640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc492030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f778e562640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0f21b0 0x5583fc491af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f778d560640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0d14d0 0x5583fc492570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f778dd61640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc492030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:47586/0 (socket says 172.21.15.96:47586) 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f778dd61640 1 -- 172.21.15.96:0/381652350 learned_addr learned my addr 172.21.15.96:0/381652350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.524+0000 7f777effd640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f77740afc50 0x7f77740b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f778dd61640 1 -- 172.21.15.96:0/381652350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0d14d0 msgr2=0x5583fc492570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f778dd61640 1 --2- 172.21.15.96:0/381652350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0d14d0 0x5583fc492570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f778dd61640 1 -- 172.21.15.96:0/381652350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0f21b0 msgr2=0x5583fc491af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f778dd61640 1 --2- 172.21.15.96:0/381652350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0f21b0 0x5583fc491af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f778dd61640 1 -- 172.21.15.96:0/381652350 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fc3bf0d0 con 0x5583fc0c92f0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f778e562640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f77740afc50 0x7f77740b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f777effd640 1 -- 172.21.15.96:0/4060963659 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f77780ff3f0 con 0x7f77881069e0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f778dd61640 1 --2- 172.21.15.96:0/381652350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc492030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f77840115c0 tx=0x7f7784011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b1be6f7bd6cece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f7773fff640 1 -- 172.21.15.96:0/381652350 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778400f640 con 0x5583fc0c92f0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f7773fff640 1 -- 172.21.15.96:0/381652350 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f778400fc90 con 0x5583fc0c92f0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.525+0000 7f7773fff640 1 -- 172.21.15.96:0/381652350 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784019ce0 con 0x5583fc0c92f0 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 -- 172.21.15.96:0/381652350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 msgr2=0x5583fc492030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f778e562640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f77740afc50 0x7f77740b2160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5583fc48ec60 tx=0x7f7780004a40 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 --2- 172.21.15.96:0/381652350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc492030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f77840115c0 tx=0x7f7784011a80 comp rx=0 tx=0).stop 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 -- 172.21.15.96:0/381652350 shutdown_connections 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 --2- 172.21.15.96:0/381652350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0d14d0 0x5583fc492570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 --2- 172.21.15.96:0/381652350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc492030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 --2- 172.21.15.96:0/381652350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0f21b0 0x5583fc491af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 -- 172.21.15.96:0/381652350 >> 172.21.15.96:0/381652350 conn(0x5583fc0e6300 msgr2=0x5583fc48e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 -- 172.21.15.96:0/381652350 shutdown_connections 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 -- 172.21.15.96:0/381652350 wait complete. 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.526+0000 7f7796e79740 1 -- start start 2024-04-25T17:14:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 1 Processor -- start 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0c92f0 0x5583fc49db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc49e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0f21b0 0x5583fc49c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583fc49ec20 con 0x5583fc0d14d0 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583fc37fb60 con 0x5583fc0c92f0 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5583fc49eda0 con 0x5583fc0f21b0 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f7796e79740 10 client.0 _ll_register_callbacks cb 0x5583fc3a42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc49e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc49e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41996/0 (socket says 172.21.15.96:41996) 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 -- 172.21.15.96:0/3337536964 learned_addr learned my addr 172.21.15.96:0/3337536964 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778e562640 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0f21b0 0x5583fc49c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778dd61640 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0c92f0 0x5583fc49db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 -- 172.21.15.96:0/3337536964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0f21b0 msgr2=0x5583fc49c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0f21b0 0x5583fc49c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 -- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0c92f0 msgr2=0x5583fc49db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0c92f0 0x5583fc49db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.527+0000 7f778d560640 1 -- 172.21.15.96:0/3337536964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fc10a210 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f778d560640 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc49e090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f77780fcbb0 tx=0x7f777803b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e989c22f0057957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f77717fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f77717fa640 1 -- 172.21.15.96:0/3337536964 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778049070 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f77717fa640 1 -- 172.21.15.96:0/3337536964 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f77780fcdd0 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778400fc90 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f77717fa640 1 -- 172.21.15.96:0/3337536964 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f77780c6fa0 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 10 client.15675 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5583fc1d6620 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f77717fa640 1 -- 172.21.15.96:0/3337536964 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f778400fc90 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f77717fa640 1 -- 172.21.15.96:0/3337536964 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f77780c38e0 con 0x5583fc0d14d0 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 10 client.15675 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 20 client.15675 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0b0d80 0x5583fc4ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 4 client.15675 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.528+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583fc406070 con 0x5583fc0b0d80 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.529+0000 7f778dd61640 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0b0d80 0x5583fc4ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.530+0000 7f77717fa640 10 client.15675 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.530+0000 7f778dd61640 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0b0d80 0x5583fc4ae7c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35fbd49d78c5a93 server_cookie=75b7918576892215 in_seq=0 out_seq=0 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f77717fa640 1 -- 172.21.15.96:0/3337536964 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583fc406070 con 0x5583fc0b0d80 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f77717fa640 10 client.15675 handle_command_reply: tid=0 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 client.15675 shutdown 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 msgr2=0x5583fc49e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc49e090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f77780fcbb0 tx=0x7f777803b640 comp rx=0 tx=0).stop 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 shutdown_connections 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0b0d80 0x5583fc4ae7c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0f21b0 0x5583fc49c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc49e090 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f77717fa640 0 client.15675 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 --2- 172.21.15.96:0/3337536964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0c92f0 0x5583fc49db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 >> 172.21.15.96:0/3337536964 conn(0x5583fc0e6300 msgr2=0x5583fc493810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 shutdown_connections 2024-04-25T17:14:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 1 -- 172.21.15.96:0/3337536964 wait complete. 2024-04-25T17:14:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.531+0000 7f7796e79740 20 client.15675 trim_cache size 0 max 16384 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.549+0000 7f7796e79740 1 Processor -- start 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.549+0000 7f7796e79740 1 -- start start 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f7796e79740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc493ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f7796e79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc4b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f7796e79740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f7796e79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77881ac8e0 con 0x5583fc0d14d0 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f7796e79740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77881af160 con 0x5583fc0f21b0 2024-04-25T17:14:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f7796e79740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77881acf50 con 0x5583fc0c92f0 2024-04-25T17:14:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778d560640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc4b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778dd61640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc493ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778d560640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc4b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42008/0 (socket says 172.21.15.96:42008) 2024-04-25T17:14:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778d560640 1 -- 172.21.15.96:0/1571562378 learned_addr learned my addr 172.21.15.96:0/1571562378 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778e562640 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778e562640 1 -- 172.21.15.96:0/1571562378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 msgr2=0x5583fc493ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778e562640 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc493ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778e562640 1 -- 172.21.15.96:0/1571562378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 msgr2=0x5583fc4b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778e562640 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc4b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778e562640 1 -- 172.21.15.96:0/1571562378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fc49b310 con 0x5583fc0f21b0 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.550+0000 7f778d560640 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc4b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.551+0000 7f778e562640 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4b44d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7780005fd0 tx=0x7f7780016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f553284077a671ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.551+0000 7f77737fe640 1 -- 172.21.15.96:0/1571562378 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780021790 con 0x5583fc0f21b0 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.551+0000 7f77737fe640 1 -- 172.21.15.96:0/1571562378 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7780021de0 con 0x5583fc0f21b0 2024-04-25T17:14:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.551+0000 7f77737fe640 1 -- 172.21.15.96:0/1571562378 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780010dc0 con 0x5583fc0f21b0 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 -- 172.21.15.96:0/1571562378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 msgr2=0x5583fc4b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4b44d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7780005fd0 tx=0x7f7780016000 comp rx=0 tx=0).stop 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 -- 172.21.15.96:0/1571562378 shutdown_connections 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4b44d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0d14d0 0x5583fc4b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 --2- 172.21.15.96:0/1571562378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0c92f0 0x5583fc493ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 -- 172.21.15.96:0/1571562378 >> 172.21.15.96:0/1571562378 conn(0x5583fc0e6300 msgr2=0x5583fc4937d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 -- 172.21.15.96:0/1571562378 shutdown_connections 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 -- 172.21.15.96:0/1571562378 wait complete. 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.552+0000 7f7796e79740 1 -- start start 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 1 Processor -- start 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0c92f0 0x5583fc4a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0d14d0 0x5583fc4a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4a1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77881ad620 con 0x5583fc0c92f0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77881aea90 con 0x5583fc0f21b0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77881adcf0 con 0x5583fc0d14d0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f7796e79740 10 client.0 _ll_register_callbacks cb 0x7f7778005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f778e562640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f778dd61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0c92f0 0x5583fc4a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f778e562640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:34594/0 (socket says 172.21.15.96:34594) 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f778e562640 1 -- 172.21.15.96:0/954880037 learned_addr learned my addr 172.21.15.96:0/954880037 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.553+0000 7f778d560640 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0d14d0 0x5583fc4a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f778e562640 1 -- 172.21.15.96:0/954880037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0d14d0 msgr2=0x5583fc4a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f778e562640 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0d14d0 0x5583fc4a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f778e562640 1 -- 172.21.15.96:0/954880037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0c92f0 msgr2=0x5583fc4a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f778e562640 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0c92f0 0x5583fc4a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f778e562640 1 -- 172.21.15.96:0/954880037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fc4a1d70 con 0x5583fc0f21b0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f778e562640 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4a1630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7780021fd0 tx=0x7f7780010630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4615446ea42ba1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f77717fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f77717fa640 1 -- 172.21.15.96:0/954880037 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780038060 con 0x5583fc0f21b0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f77717fa640 1 -- 172.21.15.96:0/954880037 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f7780010c50 con 0x5583fc0f21b0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7780021de0 con 0x5583fc0f21b0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f77717fa640 1 -- 172.21.15.96:0/954880037 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7780010e50 con 0x5583fc0f21b0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f7796e79740 10 client.25286 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.554+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5583fc514610 con 0x5583fc0f21b0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f77717fa640 1 -- 172.21.15.96:0/954880037 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7780021de0 con 0x5583fc0f21b0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f77717fa640 1 -- 172.21.15.96:0/954880037 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f778000f040 con 0x5583fc0f21b0 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f7796e79740 10 client.25286 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f7796e79740 10 client.25286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f7796e79740 20 client.25286 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f7796e79740 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0861c0 0x5583fc521750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f7796e79740 4 client.25286 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:14:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583fc3ffee0 con 0x5583fc0861c0 2024-04-25T17:14:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.555+0000 7f778dd61640 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0861c0 0x5583fc521750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.556+0000 7f77717fa640 10 client.25286 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.556+0000 7f778dd61640 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0861c0 0x5583fc521750 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b2fa92b1f81bfa server_cookie=bf8c54d12dfa8abb in_seq=0 out_seq=0 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.585+0000 7f77717fa640 1 -- 172.21.15.96:0/954880037 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583fc3ffee0 con 0x5583fc0861c0 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.585+0000 7f77717fa640 10 client.25286 handle_command_reply: tid=0 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.585+0000 7f7796e79740 1 client.25286 shutdown 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 msgr2=0x5583fc4a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4a1630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7780021fd0 tx=0x7f7780010630 comp rx=0 tx=0).stop 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 shutdown_connections 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x5583fc0861c0 0x5583fc521750 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583fc0f21b0 0x5583fc4a1630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583fc0d14d0 0x5583fc4a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 --2- 172.21.15.96:0/954880037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583fc0c92f0 0x5583fc4a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 >> 172.21.15.96:0/954880037 conn(0x5583fc0e6300 msgr2=0x5583fc49d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f77717fa640 0 client.25286 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 shutdown_connections 2024-04-25T17:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.586+0000 7f7796e79740 1 -- 172.21.15.96:0/954880037 wait complete. 2024-04-25T17:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f7796e79740 20 client.25286 trim_cache size 0 max 16384 2024-04-25T17:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f77740afc50 msgr2=0x7f77740b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f77740afc50 0x7f77740b2160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5583fc48ec60 tx=0x7f7780004a40 comp rx=0 tx=0).stop 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 msgr2=0x7f7788199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788199be0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7778006010 tx=0x7f7778004060 comp rx=0 tx=0).stop 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 shutdown_connections 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f77740afc50 0x7f77740b2160 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77881074a0 0x7f778819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77881069e0 0x7f7788199be0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 --2- 172.21.15.96:0/4060963659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7788102450 0x7f7788199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 >> 172.21.15.96:0/4060963659 conn(0x7f77880778b0 msgr2=0x7f7788100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 shutdown_connections 2024-04-25T17:14:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:15.587+0000 7f778ffc5640 1 -- 172.21.15.96:0/4060963659 wait complete. 2024-04-25T17:14:16.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:16 smithi096 ceph-mon[29131]: pgmap v124: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.3 MiB/s wr, 582 op/s 2024-04-25T17:14:16.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:16 smithi097 ceph-mon[32602]: pgmap v124: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.3 MiB/s wr, 582 op/s 2024-04-25T17:14:17.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:16 smithi171 ceph-mon[33189]: pgmap v124: 97 pgs: 97 active+clean; 56 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.3 MiB/s wr, 582 op/s 2024-04-25T17:14:17.569 INFO:tasks.workunit.client.0.smithi096.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-25T17:14:18.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:18 smithi096 ceph-mon[29131]: pgmap v125: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.4 MiB/s wr, 1.00k op/s 2024-04-25T17:14:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:18 smithi097 ceph-mon[32602]: pgmap v125: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.4 MiB/s wr, 1.00k op/s 2024-04-25T17:14:19.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:18 smithi171 ceph-mon[33189]: pgmap v125: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.4 MiB/s wr, 1.00k op/s 2024-04-25T17:14:20.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:20 smithi096 ceph-mon[29131]: pgmap v126: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 781 op/s 2024-04-25T17:14:20.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:20 smithi097 ceph-mon[32602]: pgmap v126: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 781 op/s 2024-04-25T17:14:21.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:20 smithi171 ceph-mon[33189]: pgmap v126: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 781 op/s 2024-04-25T17:14:22.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:22 smithi096 ceph-mon[29131]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.3 MiB/s wr, 1.23k op/s 2024-04-25T17:14:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:22 smithi097 ceph-mon[32602]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.3 MiB/s wr, 1.23k op/s 2024-04-25T17:14:23.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:22 smithi171 ceph-mon[33189]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.3 MiB/s wr, 1.23k op/s 2024-04-25T17:14:23.915 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x63e5d2a 2024-04-25T17:14:23.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:23 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:14:23.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:23 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:14:24.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:23 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:14:24.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:24 smithi096 ceph-mon[29131]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 873 op/s 2024-04-25T17:14:24.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:24 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:14:24.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:24 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:14:24.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:24 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:14:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:24 smithi097 ceph-mon[32602]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 873 op/s 2024-04-25T17:14:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:24 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:14:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:24 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:14:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:24 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:14:25.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:24 smithi171 ceph-mon[33189]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 873 op/s 2024-04-25T17:14:25.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:24 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:14:25.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:24 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:14:25.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:24 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:14:26.446 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x63e9b52 2024-04-25T17:14:26.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:26 smithi096 ceph-mon[29131]: pgmap v129: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 873 op/s 2024-04-25T17:14:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:26 smithi097 ceph-mon[32602]: pgmap v129: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 873 op/s 2024-04-25T17:14:27.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:26 smithi171 ceph-mon[33189]: pgmap v129: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.3 MiB/s wr, 873 op/s 2024-04-25T17:14:28.386 INFO:tasks.workunit.client.0.smithi096.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-25T17:14:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 -- 172.21.15.96:0/2673809145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724109ba0 msgr2=0x7f5724103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 --2- 172.21.15.96:0/2673809145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724109ba0 0x7f5724103740 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5714009900 tx=0x7f571402a410 comp rx=0 tx=0).stop 2024-04-25T17:14:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 -- 172.21.15.96:0/2673809145 shutdown_connections 2024-04-25T17:14:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 --2- 172.21.15.96:0/2673809145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57241042f0 0x7f57241106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 --2- 172.21.15.96:0/2673809145 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f572410a040 0x7f5724103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 --2- 172.21.15.96:0/2673809145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724109ba0 0x7f5724103740 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 -- 172.21.15.96:0/2673809145 >> 172.21.15.96:0/2673809145 conn(0x7f57240792a0 msgr2=0x7f57240796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 -- 172.21.15.96:0/2673809145 shutdown_connections 2024-04-25T17:14:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.703+0000 7f572a306640 1 -- 172.21.15.96:0/2673809145 wait complete. 2024-04-25T17:14:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.704+0000 7f572a306640 1 Processor -- start 2024-04-25T17:14:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.704+0000 7f572a306640 1 -- start start 2024-04-25T17:14:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f572a306640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241042f0 0x7f572419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f572a306640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5724109ba0 0x7f572419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f572a306640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f572410a040 0x7f572419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f572a306640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5724113d90 con 0x7f572410a040 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f572a306640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5724113c10 con 0x7f5724109ba0 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f572a306640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5724113f10 con 0x7f57241042f0 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f572410a040 0x7f572419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f572410a040 0x7f572419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57072/0 (socket says 172.21.15.96:57072) 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 -- 172.21.15.96:0/2542819868 learned_addr learned my addr 172.21.15.96:0/2542819868 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57237fe640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5724109ba0 0x7f572419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f5723fff640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241042f0 0x7f572419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 -- 172.21.15.96:0/2542819868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241042f0 msgr2=0x7f572419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241042f0 0x7f572419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 -- 172.21.15.96:0/2542819868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5724109ba0 msgr2=0x7f572419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5724109ba0 0x7f572419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 -- 172.21.15.96:0/2542819868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572419f610 con 0x7f572410a040 2024-04-25T17:14:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57288a3640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f572410a040 0x7f572419efd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f571c00eb10 tx=0x7f571c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57217fa640 1 -- 172.21.15.96:0/2542819868 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c004b20 con 0x7f572410a040 2024-04-25T17:14:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f57217fa640 1 -- 172.21.15.96:0/2542819868 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f571c004cc0 con 0x7f572410a040 2024-04-25T17:14:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.705+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572419f900 con 0x7f572410a040 2024-04-25T17:14:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.706+0000 7f57217fa640 1 -- 172.21.15.96:0/2542819868 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c010670 con 0x7f572410a040 2024-04-25T17:14:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.706+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572419fe00 con 0x7f572410a040 2024-04-25T17:14:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.711+0000 7f57311ba740 1 Processor -- start 2024-04-25T17:14:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.711+0000 7f57217fa640 1 -- 172.21.15.96:0/2542819868 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f571c005020 con 0x7f572410a040 2024-04-25T17:14:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.711+0000 7f57311ba740 1 -- start start 2024-04-25T17:14:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57311ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b9663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57311ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b964df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57311ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b965330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57311ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56506b8c7460 con 0x56506b5982f0 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57311ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56506b881590 con 0x56506b5c11b0 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57311ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56506b8c3420 con 0x56506b5a04d0 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b964df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b964df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57088/0 (socket says 172.21.15.96:57088) 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 -- 172.21.15.96:0/1470937101 learned_addr learned my addr 172.21.15.96:0/1470937101 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57237fe640 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b9663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57237fe640 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b965330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 -- 172.21.15.96:0/1470937101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 msgr2=0x56506b965330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b965330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 -- 172.21.15.96:0/1470937101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 msgr2=0x56506b9663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b9663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f5723fff640 1 -- 172.21.15.96:0/1470937101 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506b88e0d0 con 0x56506b5982f0 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.712+0000 7f57237fe640 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b9663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.713+0000 7f57217fa640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f570c0afcd0 0x7f570c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.713+0000 7f5723fff640 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b964df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5714009900 tx=0x7f5714032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9898045ee6304887 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.713+0000 7f57237fe640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f570c0afcd0 0x7f570c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.713+0000 7f57217fa640 1 -- 172.21.15.96:0/2542819868 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f571c0df690 con 0x7f572410a040 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.713+0000 7f570a7fc640 1 -- 172.21.15.96:0/1470937101 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714040070 con 0x56506b5982f0 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.713+0000 7f570a7fc640 1 -- 172.21.15.96:0/1470937101 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5714004030 con 0x56506b5982f0 2024-04-25T17:14:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.713+0000 7f570a7fc640 1 -- 172.21.15.96:0/1470937101 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714035040 con 0x56506b5982f0 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 -- 172.21.15.96:0/1470937101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 msgr2=0x56506b964df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57237fe640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f570c0afcd0 0x7f570c0b21e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x56506b965fb0 tx=0x7f57180096a0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b964df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5714009900 tx=0x7f5714032b20 comp rx=0 tx=0).stop 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 -- 172.21.15.96:0/1470937101 shutdown_connections 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b965330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b964df0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 --2- 172.21.15.96:0/1470937101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b9663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 -- 172.21.15.96:0/1470937101 >> 172.21.15.96:0/1470937101 conn(0x56506b5b5300 msgr2=0x56506b9616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 -- 172.21.15.96:0/1470937101 shutdown_connections 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.714+0000 7f57311ba740 1 -- 172.21.15.96:0/1470937101 wait complete. 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 -- start start 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 Processor -- start 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5982f0 0x56506b973db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b9742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5c11b0 0x56506b979710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56506b84eb60 con 0x56506b5c11b0 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56506b976560 con 0x56506b5a04d0 2024-04-25T17:14:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56506b979c50 con 0x56506b5982f0 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57311ba740 10 client.0 _ll_register_callbacks cb 0x56506b8732b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57288a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5c11b0 0x56506b979710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57288a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5c11b0 0x56506b979710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57094/0 (socket says 172.21.15.96:57094) 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.715+0000 7f57288a3640 1 -- 172.21.15.96:0/1231968240 learned_addr learned my addr 172.21.15.96:0/1231968240 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f5723fff640 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5982f0 0x56506b973db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57237fe640 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b9742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57288a3640 1 -- 172.21.15.96:0/1231968240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5982f0 msgr2=0x56506b973db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57288a3640 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5982f0 0x56506b973db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57288a3640 1 -- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 msgr2=0x56506b9742f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57288a3640 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b9742f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57288a3640 1 -- 172.21.15.96:0/1231968240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56506b5d9210 con 0x56506b5c11b0 2024-04-25T17:14:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57288a3640 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5c11b0 0x56506b979710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f571c0a3860 tx=0x7f571c0a7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71262de7be88b95f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57037fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57037fe640 1 -- 172.21.15.96:0/1231968240 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0c1060 con 0x56506b5c11b0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57037fe640 1 -- 172.21.15.96:0/1231968240 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f571c0900b0 con 0x56506b5c11b0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.716+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5714004030 con 0x56506b5c11b0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57037fe640 1 -- 172.21.15.96:0/1231968240 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f571c0902b0 con 0x56506b5c11b0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 10 client.15693 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56506b6a5620 con 0x56506b5c11b0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57037fe640 1 -- 172.21.15.96:0/1231968240 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5714004030 con 0x56506b5c11b0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57037fe640 1 -- 172.21.15.96:0/1231968240 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f571c005310 con 0x56506b5c11b0 2024-04-25T17:14:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 10 client.15693 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 20 client.15693 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b57fd80 0x56506b985f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 4 client.15693 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56506b8d5070 con 0x56506b57fd80 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.717+0000 7f5723fff640 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b57fd80 0x56506b985f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.718+0000 7f57037fe640 10 client.15693 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.718+0000 7f5723fff640 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b57fd80 0x56506b985f00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d05b799f09b91ff server_cookie=eed2264b925b7b6e in_seq=0 out_seq=0 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.719+0000 7f57037fe640 1 -- 172.21.15.96:0/1231968240 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56506b8d5070 con 0x56506b57fd80 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.719+0000 7f57037fe640 10 client.15693 handle_command_reply: tid=0 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.719+0000 7f57311ba740 1 client.15693 shutdown 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.719+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5c11b0 msgr2=0x56506b979710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.719+0000 7f57311ba740 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5c11b0 0x56506b979710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f571c0a3860 tx=0x7f571c0a7670 comp rx=0 tx=0).stop 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 shutdown_connections 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b57fd80 0x56506b985f00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5c11b0 0x56506b979710 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b9742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 --2- 172.21.15.96:0/1231968240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5982f0 0x56506b973db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 >> 172.21.15.96:0/1231968240 conn(0x56506b5b5300 msgr2=0x56506b966bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57037fe640 0 client.15693 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 shutdown_connections 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 1 -- 172.21.15.96:0/1231968240 wait complete. 2024-04-25T17:14:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.720+0000 7f57311ba740 20 client.15693 trim_cache size 0 max 16384 2024-04-25T17:14:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.739+0000 7f57311ba740 1 Processor -- start 2024-04-25T17:14:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.739+0000 7f57311ba740 1 -- start start 2024-04-25T17:14:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.739+0000 7f57311ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b9796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.739+0000 7f57311ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b957f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57311ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5c11b0 0x56506b971540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57311ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57241ab870 con 0x56506b5982f0 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57311ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f57241ae0d0 con 0x56506b5a04d0 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57311ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57241abec0 con 0x56506b5c11b0 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b9796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57288a3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5c11b0 0x56506b971540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b9796c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57102/0 (socket says 172.21.15.96:57102) 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f5723fff640 1 -- 172.21.15.96:0/3404122774 learned_addr learned my addr 172.21.15.96:0/3404122774 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57237fe640 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b957f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57288a3640 1 -- 172.21.15.96:0/3404122774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 msgr2=0x56506b957f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57288a3640 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b957f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57288a3640 1 -- 172.21.15.96:0/3404122774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 msgr2=0x56506b9796c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57288a3640 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b9796c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57288a3640 1 -- 172.21.15.96:0/3404122774 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506b962070 con 0x56506b5c11b0 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.740+0000 7f57288a3640 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5c11b0 0x56506b971540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f571c0a3d80 tx=0x7f571c00efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b32f96bb26023d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.741+0000 7f5709ffb640 1 -- 172.21.15.96:0/3404122774 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c0a7ee0 con 0x56506b5c11b0 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.741+0000 7f5709ffb640 1 -- 172.21.15.96:0/3404122774 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f571c0a77e0 con 0x56506b5c11b0 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.741+0000 7f5709ffb640 1 -- 172.21.15.96:0/3404122774 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571c005310 con 0x56506b5c11b0 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 -- 172.21.15.96:0/3404122774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5c11b0 msgr2=0x56506b971540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5c11b0 0x56506b971540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f571c0a3d80 tx=0x7f571c00efd0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 -- 172.21.15.96:0/3404122774 shutdown_connections 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5c11b0 0x56506b971540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5a04d0 0x56506b957f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 --2- 172.21.15.96:0/3404122774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b9796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 -- 172.21.15.96:0/3404122774 >> 172.21.15.96:0/3404122774 conn(0x56506b5b5300 msgr2=0x56506b960fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 -- 172.21.15.96:0/3404122774 shutdown_connections 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 -- 172.21.15.96:0/3404122774 wait complete. 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 -- start start 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.742+0000 7f57311ba740 1 Processor -- start 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57311ba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b97f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57311ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b97f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57311ba740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b98a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57311ba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57241ac590 con 0x56506b5982f0 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57311ba740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f57241ada00 con 0x56506b5c11b0 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57311ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57241acc60 con 0x56506b5a04d0 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57311ba740 10 client.0 _ll_register_callbacks cb 0x7f571c0a36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b97f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57237fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b97f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b97f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57118/0 (socket says 172.21.15.96:57118) 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 -- 172.21.15.96:0/275010486 learned_addr learned my addr 172.21.15.96:0/275010486 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57288a3640 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b98a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 -- 172.21.15.96:0/275010486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 msgr2=0x56506b97f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b97f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 -- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 msgr2=0x56506b98a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b98a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 -- 172.21.15.96:0/275010486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56506b98ade0 con 0x56506b5982f0 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57288a3640 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b98a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f5723fff640 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b97f290 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5714043000 tx=0x7f57140316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deae3653843c0b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.743+0000 7f57037fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57037fe640 1 -- 172.21.15.96:0/275010486 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714035040 con 0x56506b5982f0 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57037fe640 1 -- 172.21.15.96:0/275010486 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f5714004240 con 0x56506b5982f0 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f571c0a77e0 con 0x56506b5982f0 2024-04-25T17:14:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57037fe640 1 -- 172.21.15.96:0/275010486 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5714040040 con 0x56506b5982f0 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 10 client.15699 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56506b9edc40 con 0x56506b5982f0 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57037fe640 1 -- 172.21.15.96:0/275010486 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f571c0a77e0 con 0x56506b5982f0 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57037fe640 1 -- 172.21.15.96:0/275010486 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f5714003990 con 0x56506b5982f0 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 10 client.15699 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 20 client.15699 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b5551c0 0x56506b9f0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 4 client.15699 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56506b8ceee0 con 0x56506b5551c0 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.744+0000 7f57237fe640 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b5551c0 0x56506b9f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.745+0000 7f57037fe640 10 client.15699 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.745+0000 7f57237fe640 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b5551c0 0x56506b9f0010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119a229af643a583 server_cookie=da3c365265375bb1 in_seq=0 out_seq=0 2024-04-25T17:14:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57037fe640 1 -- 172.21.15.96:0/275010486 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56506b8ceee0 con 0x56506b5551c0 2024-04-25T17:14:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57037fe640 10 client.15699 handle_command_reply: tid=0 2024-04-25T17:14:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 client.15699 shutdown 2024-04-25T17:14:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 msgr2=0x56506b97f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b97f290 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5714043000 tx=0x7f57140316d0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 shutdown_connections 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56506b5551c0 0x56506b9f0010 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56506b5c11b0 0x56506b98a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56506b5a04d0 0x56506b97f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 --2- 172.21.15.96:0/275010486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56506b5982f0 0x56506b97f290 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57037fe640 0 client.15699 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 >> 172.21.15.96:0/275010486 conn(0x56506b5b5300 msgr2=0x56506b95f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 shutdown_connections 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 1 -- 172.21.15.96:0/275010486 wait complete. 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.759+0000 7f57311ba740 20 client.15699 trim_cache size 0 max 16384 2024-04-25T17:14:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:14:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:14:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:14:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f570c0afcd0 msgr2=0x7f570c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f570c0afcd0 0x7f570c0b21e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x56506b965fb0 tx=0x7f57180096a0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f572410a040 msgr2=0x7f572419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f572410a040 0x7f572419efd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f571c00eb10 tx=0x7f571c00c2d0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 shutdown_connections 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f572410a040 0x7f572419efd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f570c0afcd0 0x7f570c0b21e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5724109ba0 0x7f572419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 --2- 172.21.15.96:0/2542819868 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57241042f0 0x7f572419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 >> 172.21.15.96:0/2542819868 conn(0x7f57240792a0 msgr2=0x7f57241002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 shutdown_connections 2024-04-25T17:14:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:28.760+0000 7f572a306640 1 -- 172.21.15.96:0/2542819868 wait complete. 2024-04-25T17:14:28.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:28 smithi096 ceph-mon[29131]: pgmap v130: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.5 MiB/s wr, 1.30k op/s 2024-04-25T17:14:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:28 smithi097 ceph-mon[32602]: pgmap v130: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.5 MiB/s wr, 1.30k op/s 2024-04-25T17:14:29.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:28 smithi171 ceph-mon[33189]: pgmap v130: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.5 MiB/s wr, 1.30k op/s 2024-04-25T17:14:30.211 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x63ed6b9 2024-04-25T17:14:30.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:30 smithi096 ceph-mon[29131]: pgmap v131: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T17:14:30.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:30 smithi097 ceph-mon[32602]: pgmap v131: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T17:14:31.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:30 smithi171 ceph-mon[33189]: pgmap v131: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-04-25T17:14:32.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:32 smithi096 ceph-mon[29131]: pgmap v132: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 3.6 MiB/s wr, 1.30k op/s 2024-04-25T17:14:32.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:32 smithi097 ceph-mon[32602]: pgmap v132: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 3.6 MiB/s wr, 1.30k op/s 2024-04-25T17:14:33.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:32 smithi171 ceph-mon[33189]: pgmap v132: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 3.6 MiB/s wr, 1.30k op/s 2024-04-25T17:14:34.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:34 smithi096 ceph-mon[29131]: pgmap v133: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 848 op/s 2024-04-25T17:14:34.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:34 smithi097 ceph-mon[32602]: pgmap v133: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 848 op/s 2024-04-25T17:14:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:34 smithi171 ceph-mon[33189]: pgmap v133: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 848 op/s 2024-04-25T17:14:36.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:36 smithi096 ceph-mon[29131]: pgmap v134: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 848 op/s 2024-04-25T17:14:36.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:36 smithi097 ceph-mon[32602]: pgmap v134: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 848 op/s 2024-04-25T17:14:37.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:36 smithi171 ceph-mon[33189]: pgmap v134: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 848 op/s 2024-04-25T17:14:38.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:38 smithi096 ceph-mon[29131]: pgmap v135: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.27k op/s 2024-04-25T17:14:38.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:38 smithi097 ceph-mon[32602]: pgmap v135: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.27k op/s 2024-04-25T17:14:39.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:38 smithi171 ceph-mon[33189]: pgmap v135: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.27k op/s 2024-04-25T17:14:40.300 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x63ef9ee 2024-04-25T17:14:40.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:40 smithi096 ceph-mon[29131]: pgmap v136: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 844 op/s 2024-04-25T17:14:40.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:40 smithi097 ceph-mon[32602]: pgmap v136: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 844 op/s 2024-04-25T17:14:41.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:40 smithi171 ceph-mon[33189]: pgmap v136: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 844 op/s 2024-04-25T17:14:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.882+0000 7f4c65dd7640 1 -- 172.21.15.96:0/3819706712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108fa0 msgr2=0x7f4c60103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.882+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/3819706712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108fa0 0x7f4c60103240 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009860 tx=0x7f4c5002a340 comp rx=0 tx=0).stop 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.882+0000 7f4c65dd7640 1 -- 172.21.15.96:0/3819706712 shutdown_connections 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.882+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/3819706712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c601038b0 0x7f4c6010f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.882+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/3819706712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108fa0 0x7f4c60103240 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.882+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/3819706712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c60108b00 0x7f4c60102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.882+0000 7f4c65dd7640 1 -- 172.21.15.96:0/3819706712 >> 172.21.15.96:0/3819706712 conn(0x7f4c600fd8a0 msgr2=0x7f4c600ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.883+0000 7f4c65dd7640 1 -- 172.21.15.96:0/3819706712 shutdown_connections 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.883+0000 7f4c65dd7640 1 -- 172.21.15.96:0/3819706712 wait complete. 2024-04-25T17:14:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.883+0000 7f4c65dd7640 1 Processor -- start 2024-04-25T17:14:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c65dd7640 1 -- start start 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c65dd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c601038b0 0x7f4c6019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c65dd7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108b00 0x7f4c6019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c65dd7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c60108fa0 0x7f4c6019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c65dd7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60112cf0 con 0x7f4c60108b00 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c65dd7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60112b70 con 0x7f4c601038b0 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c65dd7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60112e70 con 0x7f4c60108fa0 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108b00 0x7f4c6019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108b00 0x7f4c6019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38518/0 (socket says 172.21.15.96:38518) 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c5effd640 1 -- 172.21.15.96:0/2743751939 learned_addr learned my addr 172.21.15.96:0/2743751939 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c601038b0 0x7f4c6019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.884+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c60108fa0 0x7f4c6019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5effd640 1 -- 172.21.15.96:0/2743751939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c60108fa0 msgr2=0x7f4c6019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5effd640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c60108fa0 0x7f4c6019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5effd640 1 -- 172.21.15.96:0/2743751939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c601038b0 msgr2=0x7f4c6019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5effd640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c601038b0 0x7f4c6019b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5effd640 1 -- 172.21.15.96:0/2743751939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c601a69f0 con 0x7f4c60108b00 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c601038b0 0x7f4c6019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5effd640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108b00 0x7f4c6019bbf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4c50035040 tx=0x7f4c50032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5cff9640 1 -- 172.21.15.96:0/2743751939 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50040020 con 0x7f4c60108b00 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c601a6c80 con 0x7f4c60108b00 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.885+0000 7f4c5cff9640 1 -- 172.21.15.96:0/2743751939 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c50030e10 con 0x7f4c60108b00 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.886+0000 7f4c5cff9640 1 -- 172.21.15.96:0/2743751939 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50037040 con 0x7f4c60108b00 2024-04-25T17:14:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.886+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c601a7170 con 0x7f4c60108b00 2024-04-25T17:14:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c5cff9640 1 -- 172.21.15.96:0/2743751939 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f4c50043020 con 0x7f4c60108b00 2024-04-25T17:14:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 Processor -- start 2024-04-25T17:14:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 -- start start 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f25315af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25316030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25316570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557f2527c460 con 0x557f24f761b0 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f25236590 con 0x557f24f554d0 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.888+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557f25278420 con 0x557f24f4d2f0 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f25315af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25316570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f25315af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38534/0 (socket says 172.21.15.96:38534) 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 -- 172.21.15.96:0/2169761137 learned_addr learned my addr 172.21.15.96:0/2169761137 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25316030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5cff9640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f4c4c0afc50 0x7f4c4c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f4c4c0afc50 0x7f4c4c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 -- 172.21.15.96:0/2169761137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 msgr2=0x557f25316030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25316030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 -- 172.21.15.96:0/2169761137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 msgr2=0x557f25316570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25316570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 -- 172.21.15.96:0/2169761137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f252430d0 con 0x557f24f761b0 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5effd640 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25316570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5cff9640 1 -- 172.21.15.96:0/2743751939 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f4c500feeb0 con 0x7f4c60108b00 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f25315af0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4c58011af0 tx=0x7f4c5800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc974deff0d46fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.889+0000 7f4c55ffb640 1 -- 172.21.15.96:0/2169761137 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58022020 con 0x557f24f761b0 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.890+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f4c4c0afc50 0x7f4c4c0b2160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c58023000 tx=0x7f4c58021000 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.890+0000 7f4c55ffb640 1 -- 172.21.15.96:0/2169761137 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c580114e0 con 0x557f24f761b0 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.890+0000 7f4c55ffb640 1 -- 172.21.15.96:0/2169761137 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5801a040 con 0x557f24f761b0 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 -- 172.21.15.96:0/2169761137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 msgr2=0x557f25315af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f25315af0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4c58011af0 tx=0x7f4c5800f3c0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 -- 172.21.15.96:0/2169761137 shutdown_connections 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25316570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25316030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/2169761137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f25315af0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 -- 172.21.15.96:0/2169761137 >> 172.21.15.96:0/2169761137 conn(0x557f24f6a300 msgr2=0x557f253122a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 -- 172.21.15.96:0/2169761137 shutdown_connections 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 -- 172.21.15.96:0/2169761137 wait complete. 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 -- start start 2024-04-25T17:14:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.891+0000 7f4c6cd67740 1 Processor -- start 2024-04-25T17:14:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25321b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f25322090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25320040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557f25322c20 con 0x557f24f554d0 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f25203b60 con 0x557f24f4d2f0 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557f25322da0 con 0x557f24f761b0 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 10 client.0 _ll_register_callbacks cb 0x557f252282b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f25322090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f25322090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38542/0 (socket says 172.21.15.96:38542) 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25321b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 -- 172.21.15.96:0/1545993613 learned_addr learned my addr 172.21.15.96:0/1545993613 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25320040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 -- 172.21.15.96:0/1545993613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 msgr2=0x557f25320040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25320040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 -- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 msgr2=0x557f25321b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25321b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 -- 172.21.15.96:0/1545993613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f24f8e210 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25321b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c5effd640 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f25322090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4c500c43e0 tx=0x7f4c50005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c64b345bcd90303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c3effd640 1 -- 172.21.15.96:0/1545993613 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500c6f40 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c3effd640 1 -- 172.21.15.96:0/1545993613 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f4c500c2ae0 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557f2528a070 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.892+0000 7f4c3effd640 1 -- 172.21.15.96:0/1545993613 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4c50030450 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 10 client.15711 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557f2505a620 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c3effd640 1 -- 172.21.15.96:0/1545993613 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557f2528a070 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c3effd640 1 -- 172.21.15.96:0/1545993613 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4c500c2ce0 con 0x557f24f554d0 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 10 client.15711 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 20 client.15711 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f34d80 0x557f253327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 4 client.15711 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f25283ee0 con 0x557f24f34d80 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f34d80 0x557f253327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.893+0000 7f4c3effd640 10 client.15711 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.894+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f34d80 0x557f253327d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92730547312f738 server_cookie=7effabb30ac646e7 in_seq=0 out_seq=0 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c3effd640 1 -- 172.21.15.96:0/1545993613 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557f25283ee0 con 0x557f24f34d80 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c3effd640 10 client.15711 handle_command_reply: tid=0 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 client.15711 shutdown 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 msgr2=0x557f25322090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f25322090 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4c500c43e0 tx=0x7f4c50005c90 comp rx=0 tx=0).stop 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 shutdown_connections 2024-04-25T17:14:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f34d80 0x557f253327d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25320040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f25322090 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c3effd640 0 client.15711 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1545993613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25321b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 >> 172.21.15.96:0/1545993613 conn(0x557f24f6a300 msgr2=0x557f25315920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 shutdown_connections 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1545993613 wait complete. 2024-04-25T17:14:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.895+0000 7f4c6cd67740 20 client.15711 trim_cache size 0 max 16384 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 Processor -- start 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 -- start start 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25319e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25337fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f253384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601ada50 con 0x557f24f761b0 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601b02d0 con 0x557f24f554d0 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601ae0c0 con 0x557f24f4d2f0 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25337fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25319e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25337fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:53078/0 (socket says 172.21.15.96:53078) 2024-04-25T17:14:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25319e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:54178/0 (socket says 172.21.15.96:54178) 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c5effd640 1 -- 172.21.15.96:0/1183516617 learned_addr learned my addr 172.21.15.96:0/1183516617 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.915+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f253384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5f7fe640 1 -- 172.21.15.96:0/1183516617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 msgr2=0x557f25337fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25337fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5f7fe640 1 -- 172.21.15.96:0/1183516617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 msgr2=0x557f253384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f253384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5f7fe640 1 -- 172.21.15.96:0/1183516617 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f253168a0 con 0x557f24f4d2f0 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5effd640 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25337fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f253384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25319e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x557f25313230 tx=0x7f4c480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f37939d16b76ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c557fa640 1 -- 172.21.15.96:0/1183516617 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48010ae0 con 0x557f24f4d2f0 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c557fa640 1 -- 172.21.15.96:0/1183516617 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c48010c80 con 0x557f24f4d2f0 2024-04-25T17:14:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.916+0000 7f4c557fa640 1 -- 172.21.15.96:0/1183516617 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4801e2b0 con 0x557f24f4d2f0 2024-04-25T17:14:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1183516617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 msgr2=0x557f25319e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25319e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x557f25313230 tx=0x7f4c480046c0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1183516617 shutdown_connections 2024-04-25T17:14:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f761b0 0x557f253384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f554d0 0x557f25337fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1183516617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f4d2f0 0x557f25319e50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1183516617 >> 172.21.15.96:0/1183516617 conn(0x557f24f6a300 msgr2=0x557f25319740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1183516617 shutdown_connections 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1183516617 wait complete. 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.917+0000 7f4c6cd67740 1 -- start start 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 1 Processor -- start 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25327360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f253278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25398b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601ae790 con 0x557f24f554d0 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601afc00 con 0x557f24f4d2f0 2024-04-25T17:14:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601aee60 con 0x557f24f761b0 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c6cd67740 10 client.0 _ll_register_callbacks cb 0x7f4c500c3610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f253278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25327360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25327360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:53082/0 (socket says 172.21.15.96:53082) 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 -- 172.21.15.96:0/1429916201 learned_addr learned my addr 172.21.15.96:0/1429916201 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f253278a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38552/0 (socket says 172.21.15.96:38552) 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5ffff640 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25398b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 -- 172.21.15.96:0/1429916201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 msgr2=0x557f25398b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25398b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 -- 172.21.15.96:0/1429916201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 msgr2=0x557f253278a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f253278a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5f7fe640 1 -- 172.21.15.96:0/1429916201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f253990a0 con 0x557f24f4d2f0 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.918+0000 7f4c5effd640 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f253278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c5f7fe640 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25327360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c48010eb0 tx=0x7f4c48010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20aa6a5ea5ee865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c3effd640 1 -- 172.21.15.96:0/1429916201 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48010ae0 con 0x557f24f4d2f0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c48010c80 con 0x557f24f4d2f0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c3effd640 1 -- 172.21.15.96:0/1429916201 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f4c4801d410 con 0x557f24f4d2f0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c3effd640 1 -- 172.21.15.96:0/1429916201 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4c4801d610 con 0x557f24f4d2f0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 10 client.25322 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557f2539a290 con 0x557f24f4d2f0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c3effd640 1 -- 172.21.15.96:0/1429916201 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c48010c80 con 0x557f24f4d2f0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c3effd640 1 -- 172.21.15.96:0/1429916201 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4c4802d030 con 0x557f24f4d2f0 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 10 client.25322 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 10 client.25322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 20 client.25322 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f0a1c0 0x557f2539c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 4 client.25322 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:14:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.919+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f251ff5f0 con 0x557f24f0a1c0 2024-04-25T17:14:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.920+0000 7f4c5effd640 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f0a1c0 0x557f2539c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.920+0000 7f4c3effd640 10 client.25322 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.920+0000 7f4c5effd640 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f0a1c0 0x557f2539c9d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5979f70c46ae317e server_cookie=fb13739007c5eb4d in_seq=0 out_seq=0 2024-04-25T17:14:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.942+0000 7f4c3effd640 1 -- 172.21.15.96:0/1429916201 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f251ff5f0 con 0x557f24f0a1c0 2024-04-25T17:14:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.942+0000 7f4c3effd640 10 client.25322 handle_command_reply: tid=0 2024-04-25T17:14:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.942+0000 7f4c6cd67740 1 client.25322 shutdown 2024-04-25T17:14:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 msgr2=0x557f25327360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25327360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c48010eb0 tx=0x7f4c48010ee0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 shutdown_connections 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x557f24f0a1c0 0x557f2539c9d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557f24f761b0 0x557f25398b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557f24f554d0 0x557f253278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 --2- 172.21.15.96:0/1429916201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f24f4d2f0 0x557f25327360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:14:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:14:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c3effd640 0 client.25322 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 >> 172.21.15.96:0/1429916201 conn(0x557f24f6a300 msgr2=0x557f2531e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 shutdown_connections 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 1 -- 172.21.15.96:0/1429916201 wait complete. 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c6cd67740 20 client.25322 trim_cache size 0 max 16384 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f4c4c0afc50 msgr2=0x7f4c4c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f4c4c0afc50 0x7f4c4c0b2160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c58023000 tx=0x7f4c58021000 comp rx=0 tx=0).stop 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108b00 msgr2=0x7f4c6019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.943+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108b00 0x7f4c6019bbf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4c50035040 tx=0x7f4c50032d50 comp rx=0 tx=0).stop 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 shutdown_connections 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c60108fa0 0x7f4c6019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c60108b00 0x7f4c6019bbf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f4c4c0afc50 0x7f4c4c0b2160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 --2- 172.21.15.96:0/2743751939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c601038b0 0x7f4c6019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 >> 172.21.15.96:0/2743751939 conn(0x7f4c600fd8a0 msgr2=0x7f4c6010e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 shutdown_connections 2024-04-25T17:14:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:41.944+0000 7f4c65dd7640 1 -- 172.21.15.96:0/2743751939 wait complete. 2024-04-25T17:14:42.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:42 smithi096 ceph-mon[29131]: pgmap v137: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 1.26k op/s 2024-04-25T17:14:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:42 smithi097 ceph-mon[32602]: pgmap v137: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 1.26k op/s 2024-04-25T17:14:43.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:42 smithi171 ceph-mon[33189]: pgmap v137: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 1.26k op/s 2024-04-25T17:14:44.986 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:44 smithi097 ceph-mon[32602]: pgmap v138: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 838 op/s 2024-04-25T17:14:45.429 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:44 smithi096 ceph-mon[29131]: pgmap v138: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 838 op/s 2024-04-25T17:14:45.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:44 smithi171 ceph-mon[33189]: pgmap v138: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 838 op/s 2024-04-25T17:14:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:46 smithi097 ceph-mon[32602]: pgmap v139: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 838 op/s 2024-04-25T17:14:47.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:46 smithi171 ceph-mon[33189]: pgmap v139: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 838 op/s 2024-04-25T17:14:47.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:46 smithi096 ceph-mon[29131]: pgmap v139: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 838 op/s 2024-04-25T17:14:48.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:47 smithi171 ceph-mon[33189]: pgmap v140: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.5 MiB/s wr, 1.25k op/s 2024-04-25T17:14:48.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:47 smithi096 ceph-mon[29131]: pgmap v140: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.5 MiB/s wr, 1.25k op/s 2024-04-25T17:14:48.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:47 smithi097 ceph-mon[32602]: pgmap v140: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.5 MiB/s wr, 1.25k op/s 2024-04-25T17:14:50.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:49 smithi096 ceph-mon[29131]: pgmap v141: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 826 op/s 2024-04-25T17:14:50.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:49 smithi097 ceph-mon[32602]: pgmap v141: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 826 op/s 2024-04-25T17:14:50.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:49 smithi171 ceph-mon[33189]: pgmap v141: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 826 op/s 2024-04-25T17:14:52.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:51 smithi096 ceph-mon[29131]: pgmap v142: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.4 MiB/s wr, 1.24k op/s 2024-04-25T17:14:52.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:51 smithi097 ceph-mon[32602]: pgmap v142: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.4 MiB/s wr, 1.24k op/s 2024-04-25T17:14:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:51 smithi171 ceph-mon[33189]: pgmap v142: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.4 MiB/s wr, 1.24k op/s 2024-04-25T17:14:54.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:53 smithi096 ceph-mon[29131]: pgmap v143: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 819 op/s 2024-04-25T17:14:54.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:53 smithi097 ceph-mon[32602]: pgmap v143: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 819 op/s 2024-04-25T17:14:54.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:53 smithi171 ceph-mon[33189]: pgmap v143: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 819 op/s 2024-04-25T17:14:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 -- 172.21.15.96:0/543536742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c108f20 msgr2=0x7fb92c102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 --2- 172.21.15.96:0/543536742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c108f20 0x7fb92c102b60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb918009860 tx=0x7fb91802a340 comp rx=0 tx=0).stop 2024-04-25T17:14:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 -- 172.21.15.96:0/543536742 shutdown_connections 2024-04-25T17:14:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 --2- 172.21.15.96:0/543536742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb92c1031d0 0x7fb92c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 --2- 172.21.15.96:0/543536742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c108f20 0x7fb92c102b60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 --2- 172.21.15.96:0/543536742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 0x7fb92c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 -- 172.21.15.96:0/543536742 >> 172.21.15.96:0/543536742 conn(0x7fb92c078150 msgr2=0x7fb92c078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 -- 172.21.15.96:0/543536742 shutdown_connections 2024-04-25T17:14:55.063 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.061+0000 7fb932ce0640 1 -- 172.21.15.96:0/543536742 wait complete. 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 Processor -- start 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 -- start start 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c1031d0 0x7fb92c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 0x7fb92c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb92c108f20 0x7fb92c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c112c70 con 0x7fb92c108a80 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c112af0 con 0x7fb92c1031d0 2024-04-25T17:14:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb932ce0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c112df0 con 0x7fb92c108f20 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 0x7fb92c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.062+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 0x7fb92c19bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34784/0 (socket says 172.21.15.96:34784) 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb92bfff640 1 -- 172.21.15.96:0/3189601926 learned_addr learned my addr 172.21.15.96:0/3189601926 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb930a7c640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c1031d0 0x7fb92c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb93127d640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb92c108f20 0x7fb92c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb92bfff640 1 -- 172.21.15.96:0/3189601926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb92c108f20 msgr2=0x7fb92c19c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb92bfff640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb92c108f20 0x7fb92c19c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb92bfff640 1 -- 172.21.15.96:0/3189601926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c1031d0 msgr2=0x7fb92c19b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb92bfff640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c1031d0 0x7fb92c19b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb92bfff640 1 -- 172.21.15.96:0/3189601926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c1a68e0 con 0x7fb92c108a80 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb930a7c640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c1031d0 0x7fb92c19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb92bfff640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 0x7fb92c19bb10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb918035040 tx=0x7fb918032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb929ffb640 1 -- 172.21.15.96:0/3189601926 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918040020 con 0x7fb92c108a80 2024-04-25T17:14:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c1a6b70 con 0x7fb92c108a80 2024-04-25T17:14:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb929ffb640 1 -- 172.21.15.96:0/3189601926 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918030450 con 0x7fb92c108a80 2024-04-25T17:14:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb929ffb640 1 -- 172.21.15.96:0/3189601926 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918037040 con 0x7fb92c108a80 2024-04-25T17:14:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.063+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c1a70b0 con 0x7fb92c108a80 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.066+0000 7fb939b91740 1 Processor -- start 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.066+0000 7fb939b91740 1 -- start start 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb929ffb640 1 -- 172.21.15.96:0/3189601926 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fb918043020 con 0x7fb92c108a80 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb939b91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fee1b0 0x556f8138daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb939b91740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fc52f0 0x556f8138e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb939b91740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8138e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb939b91740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556f812f4460 con 0x556f80fcd4d0 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb939b91740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556f812ae590 con 0x556f80fc52f0 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb939b91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556f812f0420 con 0x556f80fee1b0 2024-04-25T17:14:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8138e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb93127d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fee1b0 0x556f8138daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8138e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34798/0 (socket says 172.21.15.96:34798) 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb92bfff640 1 -- 172.21.15.96:0/1039723759 learned_addr learned my addr 172.21.15.96:0/1039723759 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb93127d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fee1b0 0x556f8138daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:36118/0 (socket says 172.21.15.96:36118) 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fc52f0 0x556f8138e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb929ffb640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb91c0afc50 0x7fb91c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb92bfff640 1 -- 172.21.15.96:0/1039723759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fee1b0 msgr2=0x556f8138daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fee1b0 0x556f8138daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb92bfff640 1 -- 172.21.15.96:0/1039723759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fc52f0 msgr2=0x556f8138e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.067+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fc52f0 0x556f8138e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb92bfff640 1 -- 172.21.15.96:0/1039723759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f812bb0d0 con 0x556f80fcd4d0 2024-04-25T17:14:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fc52f0 0x556f8138e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb93127d640 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fee1b0 0x556f8138daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:14:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb93127d640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb91c0afc50 0x7fb91c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb929ffb640 1 -- 172.21.15.96:0/3189601926 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fb9180feff0 con 0x7fb92c108a80 2024-04-25T17:14:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8138e570 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9180fc7d0 tx=0x7fb91808d780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a44d6be31dfe89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb916ffd640 1 -- 172.21.15.96:0/1039723759 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180c7080 con 0x556f80fcd4d0 2024-04-25T17:14:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb916ffd640 1 -- 172.21.15.96:0/1039723759 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918030900 con 0x556f80fcd4d0 2024-04-25T17:14:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.068+0000 7fb916ffd640 1 -- 172.21.15.96:0/1039723759 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180c6a20 con 0x556f80fcd4d0 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 -- 172.21.15.96:0/1039723759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 msgr2=0x556f8138e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb93127d640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb91c0afc50 0x7fb91c0b2160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb924009fd0 tx=0x7fb924004a40 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8138e570 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9180fc7d0 tx=0x7fb91808d780 comp rx=0 tx=0).stop 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 -- 172.21.15.96:0/1039723759 shutdown_connections 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8138e570 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fc52f0 0x556f8138e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 --2- 172.21.15.96:0/1039723759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fee1b0 0x556f8138daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 -- 172.21.15.96:0/1039723759 >> 172.21.15.96:0/1039723759 conn(0x556f80fe2300 msgr2=0x556f8138a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 -- 172.21.15.96:0/1039723759 shutdown_connections 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 -- 172.21.15.96:0/1039723759 wait complete. 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.069+0000 7fb939b91740 1 -- start start 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 1 Processor -- start 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f81398180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fcd4d0 0x556f813986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fee1b0 0x556f8139dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556f8139a9e0 con 0x556f80fee1b0 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556f8127bb60 con 0x556f80fcd4d0 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556f8139ab60 con 0x556f80fc52f0 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb939b91740 10 client.0 _ll_register_callbacks cb 0x556f812a02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb930a7c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f81398180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb93127d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fee1b0 0x556f8139dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb930a7c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f81398180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.96:36122/0 (socket says 172.21.15.96:36122) 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb930a7c640 1 -- 172.21.15.96:0/1387725657 learned_addr learned my addr 172.21.15.96:0/1387725657 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.070+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fcd4d0 0x556f813986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb930a7c640 1 -- 172.21.15.96:0/1387725657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fcd4d0 msgr2=0x556f813986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fcd4d0 0x556f813986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb930a7c640 1 -- 172.21.15.96:0/1387725657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fee1b0 msgr2=0x556f8139dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fee1b0 0x556f8139dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb930a7c640 1 -- 172.21.15.96:0/1387725657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f81006210 con 0x556f80fc52f0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f81398180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb920005fd0 tx=0x7fb92000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a13c26906344687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb90bfff640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb90bfff640 1 -- 172.21.15.96:0/1387725657 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92001a3f0 con 0x556f80fc52f0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918030900 con 0x556f80fc52f0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb90bfff640 1 -- 172.21.15.96:0/1387725657 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fb92001a590 con 0x556f80fc52f0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb90bfff640 1 -- 172.21.15.96:0/1387725657 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb920027050 con 0x556f80fc52f0 2024-04-25T17:14:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb939b91740 10 client.25300 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.071+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556f810d2620 con 0x556f80fc52f0 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb90bfff640 1 -- 172.21.15.96:0/1387725657 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918030900 con 0x556f80fc52f0 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb90bfff640 1 -- 172.21.15.96:0/1387725657 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb920022030 con 0x556f80fc52f0 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb939b91740 10 client.25300 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb939b91740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb939b91740 20 client.25300 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb939b91740 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80facd80 0x556f813aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb939b91740 4 client.25300 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f81302070 con 0x556f80facd80 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.072+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80facd80 0x556f813aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.073+0000 7fb90bfff640 10 client.25300 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.073+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80facd80 0x556f813aa4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94886c64bee96819 server_cookie=92bacb418003a77c in_seq=0 out_seq=0 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb90bfff640 1 -- 172.21.15.96:0/1387725657 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f81302070 con 0x556f80facd80 2024-04-25T17:14:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb90bfff640 10 client.25300 handle_command_reply: tid=0 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 client.25300 shutdown 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 msgr2=0x556f81398180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f81398180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb920005fd0 tx=0x7fb92000f790 comp rx=0 tx=0).stop 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 shutdown_connections 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80facd80 0x556f813aa4b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fee1b0 0x556f8139dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fcd4d0 0x556f813986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 --2- 172.21.15.96:0/1387725657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f81398180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 >> 172.21.15.96:0/1387725657 conn(0x556f80fe2300 msgr2=0x556f8138f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb90bfff640 0 client.25300 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 shutdown_connections 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 1 -- 172.21.15.96:0/1387725657 wait complete. 2024-04-25T17:14:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.074+0000 7fb939b91740 20 client.25300 trim_cache size 0 max 16384 2024-04-25T17:14:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.092+0000 7fb939b91740 1 Processor -- start 2024-04-25T17:14:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.092+0000 7fb939b91740 1 -- start start 2024-04-25T17:14:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb939b91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb939b91740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f81391870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb939b91740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f81391db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb939b91740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1ab860 con 0x556f80fcd4d0 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb939b91740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1ae0c0 con 0x556f80fee1b0 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb939b91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1abeb0 con 0x556f80fc52f0 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f81391870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f81391870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34818/0 (socket says 172.21.15.96:34818) 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 -- 172.21.15.96:0/2659025294 learned_addr learned my addr 172.21.15.96:0/2659025294 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb930a7c640 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb93127d640 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f81391db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 -- 172.21.15.96:0/2659025294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 msgr2=0x556f8139dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 -- 172.21.15.96:0/2659025294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 msgr2=0x556f81391db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f81391db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 -- 172.21.15.96:0/2659025294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8139e180 con 0x556f80fcd4d0 2024-04-25T17:14:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb93127d640 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f81391db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:14:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb92bfff640 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f81391870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9180fc910 tx=0x7fb918005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647b2cad13792052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.093+0000 7fb9167fc640 1 -- 172.21.15.96:0/2659025294 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180c6c00 con 0x556f80fcd4d0 2024-04-25T17:14:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.094+0000 7fb9167fc640 1 -- 172.21.15.96:0/2659025294 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918030900 con 0x556f80fcd4d0 2024-04-25T17:14:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.094+0000 7fb9167fc640 1 -- 172.21.15.96:0/2659025294 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180cb660 con 0x556f80fcd4d0 2024-04-25T17:14:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 -- 172.21.15.96:0/2659025294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 msgr2=0x556f81391870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f81391870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9180fc910 tx=0x7fb918005760 comp rx=0 tx=0).stop 2024-04-25T17:14:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 -- 172.21.15.96:0/2659025294 shutdown_connections 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f81391db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f81391870 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 --2- 172.21.15.96:0/2659025294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 -- 172.21.15.96:0/2659025294 >> 172.21.15.96:0/2659025294 conn(0x556f80fe2300 msgr2=0x556f81399690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 -- 172.21.15.96:0/2659025294 shutdown_connections 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 -- 172.21.15.96:0/2659025294 wait complete. 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 -- start start 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.095+0000 7fb939b91740 1 Processor -- start 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb939b91740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb939b91740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8139f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb939b91740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f8139f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb939b91740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1ac580 con 0x556f80fcd4d0 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb939b91740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1ad9f0 con 0x556f80fee1b0 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb939b91740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1acc50 con 0x556f80fc52f0 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb939b91740 10 client.0 _ll_register_callbacks cb 0x7fb920009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8139f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8139f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34822/0 (socket says 172.21.15.96:34822) 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb93127d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f8139f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 -- 172.21.15.96:0/1644518689 learned_addr learned my addr 172.21.15.96:0/1644518689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 -- 172.21.15.96:0/1644518689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 msgr2=0x556f8139ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 -- 172.21.15.96:0/1644518689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 msgr2=0x556f8139f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:14:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f8139f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 -- 172.21.15.96:0/1644518689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f81410900 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb92bfff640 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8139f490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9180c6fb0 tx=0x7fb918030e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fc8fd7b69fe761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb90bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb90bfff640 1 -- 172.21.15.96:0/1644518689 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180c6c00 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.096+0000 7fb90bfff640 1 -- 172.21.15.96:0/1644518689 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7fb9180fc940 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918030900 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb90bfff640 1 -- 172.21.15.96:0/1644518689 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb9180fcb40 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 10 client.15735 fetch_fsmap learned FSMap version 20 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556f81411b90 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb90bfff640 1 -- 172.21.15.96:0/1644518689 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918030900 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb90bfff640 1 -- 172.21.15.96:0/1644518689 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb91808e3d0 con 0x556f80fcd4d0 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 10 client.15735 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:14:55.102 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 20 client.15735 populate_metadata read hostname 'smithi096' 2024-04-25T17:14:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80f821c0 0x556f81414040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:14:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 4 client.15735 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:14:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.097+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f812fbee0 con 0x556f80f821c0 2024-04-25T17:14:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.098+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80f821c0 0x556f81414040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:14:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.098+0000 7fb90bfff640 10 client.15735 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.098+0000 7fb930a7c640 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80f821c0 0x556f81414040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a6e8fea37d4eaa server_cookie=3e1a4f226a1a4a79 in_seq=0 out_seq=0 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.116+0000 7fb90bfff640 1 -- 172.21.15.96:0/1644518689 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f812fbee0 con 0x556f80f821c0 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.116+0000 7fb90bfff640 10 client.15735 handle_command_reply: tid=0 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 client.15735 shutdown 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 msgr2=0x556f8139f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8139f490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9180c6fb0 tx=0x7fb918030e90 comp rx=0 tx=0).stop 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 shutdown_connections 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x556f80f821c0 0x556f81414040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556f80fee1b0 0x556f8139f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556f80fcd4d0 0x556f8139f490 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb90bfff640 0 client.15735 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:14:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 --2- 172.21.15.96:0/1644518689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556f80fc52f0 0x556f8139ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 >> 172.21.15.96:0/1644518689 conn(0x556f80fe2300 msgr2=0x556f8138fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 shutdown_connections 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 1 -- 172.21.15.96:0/1644518689 wait complete. 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.117+0000 7fb939b91740 20 client.15735 trim_cache size 0 max 16384 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:14:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb91c0afc50 msgr2=0x7fb91c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb91c0afc50 0x7fb91c0b2160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb924009fd0 tx=0x7fb924004a40 comp rx=0 tx=0).stop 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 msgr2=0x7fb92c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 0x7fb92c19bb10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb918035040 tx=0x7fb918032dc0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 shutdown_connections 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb92c108f20 0x7fb92c19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb92c108a80 0x7fb92c19bb10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fb91c0afc50 0x7fb91c0b2160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 --2- 172.21.15.96:0/3189601926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb92c1031d0 0x7fb92c19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:14:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 >> 172.21.15.96:0/3189601926 conn(0x7fb92c078150 msgr2=0x7fb92c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:14:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 shutdown_connections 2024-04-25T17:14:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:14:55.118+0000 7fb932ce0640 1 -- 172.21.15.96:0/3189601926 wait complete. 2024-04-25T17:14:56.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:55 smithi096 ceph-mon[29131]: pgmap v144: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 819 op/s 2024-04-25T17:14:56.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:55 smithi097 ceph-mon[32602]: pgmap v144: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 819 op/s 2024-04-25T17:14:56.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:55 smithi171 ceph-mon[33189]: pgmap v144: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 819 op/s 2024-04-25T17:14:58.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:57 smithi096 ceph-mon[29131]: pgmap v145: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.4 MiB/s wr, 1.22k op/s 2024-04-25T17:14:58.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:57 smithi097 ceph-mon[32602]: pgmap v145: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.4 MiB/s wr, 1.22k op/s 2024-04-25T17:14:58.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:57 smithi171 ceph-mon[33189]: pgmap v145: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.4 MiB/s wr, 1.22k op/s 2024-04-25T17:15:00.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:14:59 smithi097 ceph-mon[32602]: pgmap v146: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 812 op/s 2024-04-25T17:15:00.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:14:59 smithi171 ceph-mon[33189]: pgmap v146: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 812 op/s 2024-04-25T17:15:00.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:14:59 smithi096 ceph-mon[29131]: pgmap v146: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.2 MiB/s wr, 812 op/s 2024-04-25T17:15:02.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:01 smithi097 ceph-mon[32602]: pgmap v147: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.3 MiB/s wr, 1.20k op/s 2024-04-25T17:15:02.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:01 smithi171 ceph-mon[33189]: pgmap v147: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.3 MiB/s wr, 1.20k op/s 2024-04-25T17:15:02.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:01 smithi096 ceph-mon[29131]: pgmap v147: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.3 MiB/s wr, 1.20k op/s 2024-04-25T17:15:02.987 INFO:tasks.workunit.client.0.smithi096.stdout:All 50000 operations completed A-OK! 2024-04-25T17:15:02.989 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T17:15:02.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T17:15:04.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:03 smithi097 ceph-mon[32602]: pgmap v148: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 793 op/s 2024-04-25T17:15:04.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:03 smithi171 ceph-mon[33189]: pgmap v148: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 793 op/s 2024-04-25T17:15:04.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:03 smithi096 ceph-mon[29131]: pgmap v148: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 793 op/s 2024-04-25T17:15:06.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:05 smithi097 ceph-mon[32602]: pgmap v149: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 793 op/s 2024-04-25T17:15:06.293 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:05 smithi171 ceph-mon[33189]: pgmap v149: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 793 op/s 2024-04-25T17:15:06.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:05 smithi096 ceph-mon[29131]: pgmap v149: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.2 MiB/s wr, 793 op/s 2024-04-25T17:15:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 -- 172.21.15.96:0/3715289427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31241072c0 msgr2=0x7f31241076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 --2- 172.21.15.96:0/3715289427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31241072c0 0x7f31241076d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3118009900 tx=0x7f311802a410 comp rx=0 tx=0).stop 2024-04-25T17:15:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 -- 172.21.15.96:0/3715289427 shutdown_connections 2024-04-25T17:15:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 --2- 172.21.15.96:0/3715289427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3124107d40 0x7f312410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 --2- 172.21.15.96:0/3715289427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31241072c0 0x7f31241076d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 --2- 172.21.15.96:0/3715289427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3124103400 0x7f3124103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 -- 172.21.15.96:0/3715289427 >> 172.21.15.96:0/3715289427 conn(0x7f31240fd8a0 msgr2=0x7f31240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:15:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 -- 172.21.15.96:0/3715289427 shutdown_connections 2024-04-25T17:15:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 -- 172.21.15.96:0/3715289427 wait complete. 2024-04-25T17:15:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.219+0000 7f312a6e0640 1 Processor -- start 2024-04-25T17:15:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f312a6e0640 1 -- start start 2024-04-25T17:15:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f312a6e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3124103400 0x7f312419b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f312a6e0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31241072c0 0x7f312419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f312a6e0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3124107d40 0x7f31241a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f312a6e0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3124112c40 con 0x7f3124107d40 2024-04-25T17:15:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f312a6e0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3124112ac0 con 0x7f3124103400 2024-04-25T17:15:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f312a6e0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3124112dc0 con 0x7f31241072c0 2024-04-25T17:15:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3124107d40 0x7f31241a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3124107d40 0x7f31241a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47034/0 (socket says 172.21.15.96:47034) 2024-04-25T17:15:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 -- 172.21.15.96:0/4252194059 learned_addr learned my addr 172.21.15.96:0/4252194059 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:15:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3123fff640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3124103400 0x7f312419b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f31237fe640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31241072c0 0x7f312419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 -- 172.21.15.96:0/4252194059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31241072c0 msgr2=0x7f312419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31241072c0 0x7f312419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 -- 172.21.15.96:0/4252194059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3124103400 msgr2=0x7f312419b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3124103400 0x7f312419b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3128c7d640 1 -- 172.21.15.96:0/4252194059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312419c300 con 0x7f3124107d40 2024-04-25T17:15:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f31237fe640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31241072c0 0x7f312419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:15:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.220+0000 7f3123fff640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3124103400 0x7f312419b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:15:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.221+0000 7f3128c7d640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3124107d40 0x7f31241a65c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f311c00eb10 tx=0x7f311c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:15:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.221+0000 7f31217fa640 1 -- 172.21.15.96:0/4252194059 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c004b20 con 0x7f3124107d40 2024-04-25T17:15:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.221+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31241a6bc0 con 0x7f3124107d40 2024-04-25T17:15:08.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.221+0000 7f31217fa640 1 -- 172.21.15.96:0/4252194059 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311c004cc0 con 0x7f3124107d40 2024-04-25T17:15:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.221+0000 7f31217fa640 1 -- 172.21.15.96:0/4252194059 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c010670 con 0x7f3124107d40 2024-04-25T17:15:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.221+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31241a70e0 con 0x7f3124107d40 2024-04-25T17:15:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.223+0000 7f3131591740 1 Processor -- start 2024-04-25T17:15:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.223+0000 7f31217fa640 1 -- 172.21.15.96:0/4252194059 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f311c005020 con 0x7f3124107d40 2024-04-25T17:15:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.223+0000 7f3131591740 1 -- start start 2024-04-25T17:15:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3131591740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5a31b0 0x56257c942af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3131591740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c943030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3131591740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c943570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3131591740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56257c8a9460 con 0x56257c57a2f0 2024-04-25T17:15:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3131591740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56257c863590 con 0x56257c5a31b0 2024-04-25T17:15:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3131591740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56257c8a5420 con 0x56257c5824d0 2024-04-25T17:15:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c943030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c943030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47046/0 (socket says 172.21.15.96:47046) 2024-04-25T17:15:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f31237fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5a31b0 0x56257c942af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 -- 172.21.15.96:0/3089960056 learned_addr learned my addr 172.21.15.96:0/3089960056 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:15:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f31217fa640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f310c0afc50 0x7f310c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 -- 172.21.15.96:0/3089960056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 msgr2=0x56257c943570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c943570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 -- 172.21.15.96:0/3089960056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5a31b0 msgr2=0x56257c942af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5a31b0 0x56257c942af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 -- 172.21.15.96:0/3089960056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257c8700d0 con 0x56257c57a2f0 2024-04-25T17:15:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f31237fe640 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5a31b0 0x56257c942af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T17:15:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3128c7d640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f310c0afc50 0x7f310c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f3123fff640 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c943030 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3114010450 tx=0x7f3114010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d641d15ba0cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:15:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f31217fa640 1 -- 172.21.15.96:0/4252194059 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f311c0df650 con 0x7f3124107d40 2024-04-25T17:15:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f310a7fc640 1 -- 172.21.15.96:0/3089960056 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114011760 con 0x56257c57a2f0 2024-04-25T17:15:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f310a7fc640 1 -- 172.21.15.96:0/3089960056 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3114011db0 con 0x56257c57a2f0 2024-04-25T17:15:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.224+0000 7f310a7fc640 1 -- 172.21.15.96:0/3089960056 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311401a380 con 0x56257c57a2f0 2024-04-25T17:15:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 -- 172.21.15.96:0/3089960056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 msgr2=0x56257c943030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c943030 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3114010450 tx=0x7f3114010910 comp rx=0 tx=0).stop 2024-04-25T17:15:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3128c7d640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f310c0afc50 0x7f310c0b2160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f311c0dce60 tx=0x7f311c00efd0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:15:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 -- 172.21.15.96:0/3089960056 shutdown_connections 2024-04-25T17:15:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c943570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c943030 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 --2- 172.21.15.96:0/3089960056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5a31b0 0x56257c942af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 -- 172.21.15.96:0/3089960056 >> 172.21.15.96:0/3089960056 conn(0x56257c597300 msgr2=0x56257c93f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:15:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 -- 172.21.15.96:0/3089960056 shutdown_connections 2024-04-25T17:15:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.225+0000 7f3131591740 1 -- 172.21.15.96:0/3089960056 wait complete. 2024-04-25T17:15:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 -- start start 2024-04-25T17:15:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 Processor -- start 2024-04-25T17:15:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c94eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c94f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5a31b0 0x56257c94cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56257c94fc20 con 0x56257c57a2f0 2024-04-25T17:15:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56257c830b60 con 0x56257c5824d0 2024-04-25T17:15:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56257c94fda0 con 0x56257c5a31b0 2024-04-25T17:15:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3131591740 10 client.0 _ll_register_callbacks cb 0x56257c8552b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:15:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f31237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c94eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f31237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c94eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47048/0 (socket says 172.21.15.96:47048) 2024-04-25T17:15:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f31237fe640 1 -- 172.21.15.96:0/2517822082 learned_addr learned my addr 172.21.15.96:0/2517822082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:15:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.226+0000 7f3123fff640 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c94f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31237fe640 1 -- 172.21.15.96:0/2517822082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5a31b0 msgr2=0x56257c94cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31237fe640 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5a31b0 0x56257c94cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31237fe640 1 -- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 msgr2=0x56257c94f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31237fe640 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c94f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31237fe640 1 -- 172.21.15.96:0/2517822082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257c5bb210 con 0x56257c57a2f0 2024-04-25T17:15:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31237fe640 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c94eb00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5af991ac01e8ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:15:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31037fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31037fe640 1 -- 172.21.15.96:0/2517822082 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118041070 con 0x56257c57a2f0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31037fe640 1 -- 172.21.15.96:0/2517822082 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f3118004450 con 0x56257c57a2f0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3114011db0 con 0x56257c57a2f0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31037fe640 1 -- 172.21.15.96:0/2517822082 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3118035070 con 0x56257c57a2f0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f3131591740 10 client.15747 fetch_fsmap learned FSMap version 20 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56257c687620 con 0x56257c57a2f0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31037fe640 1 -- 172.21.15.96:0/2517822082 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3114011db0 con 0x56257c57a2f0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.227+0000 7f31037fe640 1 -- 172.21.15.96:0/2517822082 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3118004b10 con 0x56257c57a2f0 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3131591740 10 client.15747 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3131591740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3131591740 20 client.15747 populate_metadata read hostname 'smithi096' 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3131591740 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c561d80 0x56257c95f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3131591740 4 client.15747 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T17:15:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56257c8b7070 con 0x56257c561d80 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3123fff640 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c561d80 0x56257c95f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f31037fe640 10 client.15747 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.228+0000 7f3123fff640 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c561d80 0x56257c95f720 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adb2f57b495be790 server_cookie=62c647e9041eb1ea in_seq=0 out_seq=0 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f31037fe640 1 -- 172.21.15.96:0/2517822082 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56257c8b7070 con 0x56257c561d80 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f31037fe640 10 client.15747 handle_command_reply: tid=0 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 client.15747 shutdown 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 msgr2=0x56257c94eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c94eb00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118031810 comp rx=0 tx=0).stop 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 shutdown_connections 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c561d80 0x56257c95f720 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5a31b0 0x56257c94cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c94f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f31037fe640 0 client.15747 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:15:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 --2- 172.21.15.96:0/2517822082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c94eb00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 >> 172.21.15.96:0/2517822082 conn(0x56257c597300 msgr2=0x56257c944810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:15:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 shutdown_connections 2024-04-25T17:15:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 1 -- 172.21.15.96:0/2517822082 wait complete. 2024-04-25T17:15:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.230+0000 7f3131591740 20 client.15747 trim_cache size 0 max 16384 2024-04-25T17:15:08.256 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:07 smithi097 ceph-mon[32602]: pgmap v150: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.4 MiB/s wr, 1.00k op/s 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 Processor -- start 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 -- start start 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c561d80 0x56257c944eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c57a2f0 0x56257c9467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c946ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f31241ad960 con 0x56257c561d80 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f31241b01e0 con 0x56257c5824d0 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3131591740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f31241adfd0 con 0x56257c57a2f0 2024-04-25T17:15:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c561d80 0x56257c944eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c561d80 0x56257c944eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47058/0 (socket says 172.21.15.96:47058) 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 -- 172.21.15.96:0/451276768 learned_addr learned my addr 172.21.15.96:0/451276768 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c946ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f3123fff640 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c57a2f0 0x56257c9467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 -- 172.21.15.96:0/451276768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c57a2f0 msgr2=0x56257c9467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c57a2f0 0x56257c9467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 -- 172.21.15.96:0/451276768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 msgr2=0x56257c946ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c946ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 -- 172.21.15.96:0/451276768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257c939550 con 0x56257c561d80 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.246+0000 7f31237fe640 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c561d80 0x56257c944eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3118009850 tx=0x7f311802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29f974a6c8ff183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3109ffb640 1 -- 172.21.15.96:0/451276768 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118000e60 con 0x56257c561d80 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3109ffb640 1 -- 172.21.15.96:0/451276768 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3118004450 con 0x56257c561d80 2024-04-25T17:15:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3109ffb640 1 -- 172.21.15.96:0/451276768 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118041070 con 0x56257c561d80 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3131591740 1 -- 172.21.15.96:0/451276768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c561d80 msgr2=0x56257c944eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3131591740 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c561d80 0x56257c944eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3118009850 tx=0x7f311802a410 comp rx=0 tx=0).stop 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3131591740 1 -- 172.21.15.96:0/451276768 shutdown_connections 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3131591740 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c5824d0 0x56257c946ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3131591740 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c57a2f0 0x56257c9467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.247+0000 7f3131591740 1 --2- 172.21.15.96:0/451276768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c561d80 0x56257c944eb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 -- 172.21.15.96:0/451276768 >> 172.21.15.96:0/451276768 conn(0x56257c597300 msgr2=0x56257c9447a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 -- 172.21.15.96:0/451276768 shutdown_connections 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 -- 172.21.15.96:0/451276768 wait complete. 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 -- start start 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 Processor -- start 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c561d80 0x56257c9bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c9c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c9524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f31241ae6a0 con 0x56257c57a2f0 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f31241afb10 con 0x56257c561d80 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f31241aed70 con 0x56257c5824d0 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3131591740 10 client.0 _ll_register_callbacks cb 0x7f3118009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3123fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c9c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.248+0000 7f3123fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c9c0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47066/0 (socket says 172.21.15.96:47066) 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3123fff640 1 -- 172.21.15.96:0/1848777426 learned_addr learned my addr 172.21.15.96:0/1848777426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f31237fe640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c561d80 0x56257c9bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f31237fe640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c9524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3123fff640 1 -- 172.21.15.96:0/1848777426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 msgr2=0x56257c9524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3123fff640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c9524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3123fff640 1 -- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c561d80 msgr2=0x56257c9bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3123fff640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c561d80 0x56257c9bfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:08.264 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3123fff640 1 -- 172.21.15.96:0/1848777426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257c952af0 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f31237fe640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c9524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3123fff640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c9c0370 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3114006010 tx=0x7f311400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e3d192801b4f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f31037fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f31037fe640 1 -- 172.21.15.96:0/1848777426 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311401c500 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f31037fe640 1 -- 172.21.15.96:0/1848777426 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8498+0+0 (secure 0 0 0) 0x7f311401c6a0 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3118004450 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f31037fe640 1 -- 172.21.15.96:0/1848777426 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3114038050 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3131591740 10 client.15753 fetch_fsmap learned FSMap version 20 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.249+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56257c9c74e0 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f31037fe640 1 -- 172.21.15.96:0/1848777426 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3118004450 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f31037fe640 1 -- 172.21.15.96:0/1848777426 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3114003990 con 0x56257c57a2f0 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f3131591740 10 client.15753 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f3131591740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f3131591740 20 client.15753 populate_metadata read hostname 'smithi096' 2024-04-25T17:15:08.265 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f3131591740 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c5a31b0 0x56257c9c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:15:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f3131591740 4 client.15753 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T17:15:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 --> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56257c8b0ee0 con 0x56257c5a31b0 2024-04-25T17:15:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f31237fe640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c5a31b0 0x56257c9c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:15:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f31037fe640 10 client.15753 ms_handle_connect on v2:172.21.15.97:6838/3298672156 2024-04-25T17:15:08.266 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:08.250+0000 7f31237fe640 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c5a31b0 0x56257c9c9960 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfed0f3f095928a server_cookie=76602369e9eb3c4f in_seq=0 out_seq=0 2024-04-25T17:15:08.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:07 smithi171 ceph-mon[33189]: pgmap v150: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.4 MiB/s wr, 1.00k op/s 2024-04-25T17:15:08.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:07 smithi096 ceph-mon[29131]: pgmap v150: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.4 MiB/s wr, 1.00k op/s 2024-04-25T17:15:10.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:09 smithi097 ceph-mon[32602]: pgmap v151: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 603 op/s 2024-04-25T17:15:10.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:09 smithi171 ceph-mon[33189]: pgmap v151: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 603 op/s 2024-04-25T17:15:10.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:09 smithi096 ceph-mon[29131]: pgmap v151: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.3 MiB/s wr, 603 op/s 2024-04-25T17:15:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f31037fe640 1 -- 172.21.15.96:0/1848777426 <== mds.0 v2:172.21.15.97:6838/3298672156 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56257c8b0ee0 con 0x56257c5a31b0 2024-04-25T17:15:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f31037fe640 10 client.15753 handle_command_reply: tid=0 2024-04-25T17:15:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f3131591740 1 client.15753 shutdown 2024-04-25T17:15:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 msgr2=0x56257c9c0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f3131591740 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c9c0370 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3114006010 tx=0x7f311400ba40 comp rx=0 tx=0).stop 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 shutdown_connections 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f3131591740 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:6838/3298672156,v1:172.21.15.97:6839/3298672156] conn(0x56257c5a31b0 0x56257c9c9960 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f3131591740 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56257c5824d0 0x56257c9524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f3131591740 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56257c57a2f0 0x56257c9c0370 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.383+0000 7f3131591740 1 --2- 172.21.15.96:0/1848777426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56257c561d80 0x56257c9bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.383+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 >> 172.21.15.96:0/1848777426 conn(0x56257c597300 msgr2=0x56257c94e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.382+0000 7f31037fe640 0 client.15753 ms_handle_reset on v2:172.21.15.97:6838/3298672156 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.383+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 shutdown_connections 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.383+0000 7f3131591740 1 -- 172.21.15.96:0/1848777426 wait complete. 2024-04-25T17:15:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.383+0000 7f3131591740 20 client.15753 trim_cache size 0 max 16384 2024-04-25T17:15:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T17:15:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T17:15:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T17:15:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f310c0afc50 msgr2=0x7f310c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f310c0afc50 0x7f310c0b2160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f311c0dce60 tx=0x7f311c00efd0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3124107d40 msgr2=0x7f31241a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3124107d40 0x7f31241a65c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f311c00eb10 tx=0x7f311c00c2d0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 shutdown_connections 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3124107d40 0x7f31241a65c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31241072c0 0x7f312419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f310c0afc50 0x7f310c0b2160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 --2- 172.21.15.96:0/4252194059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3124103400 0x7f312419b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 >> 172.21.15.96:0/4252194059 conn(0x7f31240fd8a0 msgr2=0x7f31241056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:15:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 shutdown_connections 2024-04-25T17:15:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T17:15:11.384+0000 7f312a6e0640 1 -- 172.21.15.96:0/4252194059 wait complete. 2024-04-25T17:15:12.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:11 smithi097 ceph-mon[32602]: pgmap v152: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.5 MiB/s wr, 799 op/s 2024-04-25T17:15:12.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:11 smithi171 ceph-mon[33189]: pgmap v152: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.5 MiB/s wr, 799 op/s 2024-04-25T17:15:12.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:11 smithi096 ceph-mon[29131]: pgmap v152: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.5 MiB/s wr, 799 op/s 2024-04-25T17:15:14.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:13 smithi097 ceph-mon[32602]: pgmap v153: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 408 op/s 2024-04-25T17:15:14.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:13 smithi171 ceph-mon[33189]: pgmap v153: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 408 op/s 2024-04-25T17:15:14.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:13 smithi096 ceph-mon[29131]: pgmap v153: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 408 op/s 2024-04-25T17:15:15.374 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-25T17:15:15.375 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T17:15:15.691 DEBUG:teuthology.parallel:result is None 2024-04-25T17:15:15.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T17:15:15.722 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T17:15:15.726 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T17:15:15.736 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:15:15.770 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T17:15:15.779 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T17:15:15.779 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T17:15:15.780 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:15:15.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T17:15:15.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T17:15:16.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:15 smithi097 ceph-mon[32602]: pgmap v154: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 408 op/s 2024-04-25T17:15:16.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:15 smithi171 ceph-mon[33189]: pgmap v154: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 408 op/s 2024-04-25T17:15:16.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:15 smithi096 ceph-mon[29131]: pgmap v154: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 408 op/s 2024-04-25T17:15:18.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:17 smithi097 ceph-mon[32602]: pgmap v155: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.2 MiB/s wr, 607 op/s 2024-04-25T17:15:18.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:17 smithi171 ceph-mon[33189]: pgmap v155: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.2 MiB/s wr, 607 op/s 2024-04-25T17:15:18.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:17 smithi096 ceph-mon[29131]: pgmap v155: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.2 MiB/s wr, 607 op/s 2024-04-25T17:15:20.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:19 smithi097 ceph-mon[32602]: pgmap v156: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 394 op/s 2024-04-25T17:15:20.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:19 smithi171 ceph-mon[33189]: pgmap v156: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 394 op/s 2024-04-25T17:15:20.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:19 smithi096 ceph-mon[29131]: pgmap v156: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 394 op/s 2024-04-25T17:15:22.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:21 smithi097 ceph-mon[32602]: pgmap v157: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-04-25T17:15:22.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:21 smithi171 ceph-mon[33189]: pgmap v157: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-04-25T17:15:22.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:21 smithi096 ceph-mon[29131]: pgmap v157: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 407 op/s 2024-04-25T17:15:24.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:23 smithi097 ceph-mon[32602]: pgmap v158: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 896 KiB/s wr, 211 op/s 2024-04-25T17:15:24.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:23 smithi171 ceph-mon[33189]: pgmap v158: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 896 KiB/s wr, 211 op/s 2024-04-25T17:15:24.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:23 smithi096 ceph-mon[29131]: pgmap v158: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 896 KiB/s wr, 211 op/s 2024-04-25T17:15:25.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:24 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:15:25.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:24 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:15:25.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:24 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:15:25.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:24 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:15:25.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:24 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:15:25.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:24 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:15:25.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:24 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:15:25.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:24 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:15:25.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:24 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:15:25.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:24 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:15:25.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:24 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:15:25.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:24 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:15:26.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:25 smithi171 ceph-mon[33189]: pgmap v159: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 896 KiB/s wr, 211 op/s 2024-04-25T17:15:26.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:25 smithi096 ceph-mon[29131]: pgmap v159: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 896 KiB/s wr, 211 op/s 2024-04-25T17:15:26.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:25 smithi097 ceph-mon[32602]: pgmap v159: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 896 KiB/s wr, 211 op/s 2024-04-25T17:15:28.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:27 smithi171 ceph-mon[33189]: pgmap v160: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 897 KiB/s wr, 213 op/s 2024-04-25T17:15:28.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:27 smithi096 ceph-mon[29131]: pgmap v160: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 897 KiB/s wr, 213 op/s 2024-04-25T17:15:28.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:27 smithi097 ceph-mon[32602]: pgmap v160: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 897 KiB/s wr, 213 op/s 2024-04-25T17:15:30.242 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:29 smithi096 ceph-mon[29131]: pgmap v161: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T17:15:30.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:29 smithi171 ceph-mon[33189]: pgmap v161: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T17:15:30.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:29 smithi097 ceph-mon[32602]: pgmap v161: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-25T17:15:32.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:31 smithi171 ceph-mon[33189]: pgmap v162: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-25T17:15:32.286 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:31 smithi097 ceph-mon[32602]: pgmap v162: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-25T17:15:32.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:31 smithi096 ceph-mon[29131]: pgmap v162: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-25T17:15:34.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:33 smithi171 ceph-mon[33189]: pgmap v163: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:15:34.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:33 smithi096 ceph-mon[29131]: pgmap v163: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:15:34.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:33 smithi097 ceph-mon[32602]: pgmap v163: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:15:36.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:36 smithi171 ceph-mon[33189]: pgmap v164: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:15:36.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:36 smithi096 ceph-mon[29131]: pgmap v164: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:15:36.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:36 smithi097 ceph-mon[32602]: pgmap v164: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T17:15:38.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:38 smithi171 ceph-mon[33189]: pgmap v165: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-25T17:15:38.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:38 smithi096 ceph-mon[29131]: pgmap v165: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-25T17:15:38.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:38 smithi097 ceph-mon[32602]: pgmap v165: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-25T17:15:40.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:40 smithi171 ceph-mon[33189]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-25T17:15:40.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:40 smithi096 ceph-mon[29131]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-25T17:15:40.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:40 smithi097 ceph-mon[32602]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-25T17:15:42.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:42 smithi171 ceph-mon[33189]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-25T17:15:42.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:42 smithi096 ceph-mon[29131]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-25T17:15:42.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:42 smithi097 ceph-mon[32602]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-25T17:15:44.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:44 smithi171 ceph-mon[33189]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:44.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:44 smithi096 ceph-mon[29131]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:44.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:44 smithi097 ceph-mon[32602]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:46.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:46 smithi171 ceph-mon[33189]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:46.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:46 smithi096 ceph-mon[29131]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:46.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:46 smithi097 ceph-mon[32602]: pgmap v169: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:48.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:48 smithi171 ceph-mon[33189]: pgmap v170: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:48.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:48 smithi096 ceph-mon[29131]: pgmap v170: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:48 smithi097 ceph-mon[32602]: pgmap v170: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 853 B/s wr, 2 op/s 2024-04-25T17:15:50.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:50 smithi171 ceph-mon[33189]: pgmap v171: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-25T17:15:50.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:50 smithi096 ceph-mon[29131]: pgmap v171: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-25T17:15:50.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:50 smithi097 ceph-mon[32602]: pgmap v171: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-25T17:15:52.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:52 smithi171 ceph-mon[33189]: pgmap v172: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-25T17:15:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:52 smithi096 ceph-mon[29131]: pgmap v172: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-25T17:15:52.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:52 smithi097 ceph-mon[32602]: pgmap v172: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-25T17:15:54.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:54 smithi171 ceph-mon[33189]: pgmap v173: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:15:54.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:54 smithi096 ceph-mon[29131]: pgmap v173: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:15:54.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:54 smithi097 ceph-mon[32602]: pgmap v173: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:15:56.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:56 smithi096 ceph-mon[29131]: pgmap v174: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:15:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:56 smithi097 ceph-mon[32602]: pgmap v174: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:15:56.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:56 smithi171 ceph-mon[33189]: pgmap v174: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:15:58.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:15:58 smithi096 ceph-mon[29131]: pgmap v175: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:15:58.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:15:58 smithi097 ceph-mon[32602]: pgmap v175: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:15:58.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:15:58 smithi171 ceph-mon[33189]: pgmap v175: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:00.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:00 smithi096 ceph-mon[29131]: pgmap v176: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:00 smithi097 ceph-mon[32602]: pgmap v176: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:00.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:00 smithi171 ceph-mon[33189]: pgmap v176: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:02.332 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:02 smithi171 ceph-mon[33189]: pgmap v177: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:02.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:02 smithi096 ceph-mon[29131]: pgmap v177: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:02 smithi097 ceph-mon[32602]: pgmap v177: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:04.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:04 smithi096 ceph-mon[29131]: pgmap v178: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:04 smithi097 ceph-mon[32602]: pgmap v178: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:04.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:04 smithi171 ceph-mon[33189]: pgmap v178: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:06.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:06 smithi096 ceph-mon[29131]: pgmap v179: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:06 smithi097 ceph-mon[32602]: pgmap v179: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:06.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:06 smithi171 ceph-mon[33189]: pgmap v179: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:08.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:08 smithi096 ceph-mon[29131]: pgmap v180: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:08 smithi097 ceph-mon[32602]: pgmap v180: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:08.530 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:08 smithi171 ceph-mon[33189]: pgmap v180: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:10.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:10 smithi096 ceph-mon[29131]: pgmap v181: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:10 smithi097 ceph-mon[32602]: pgmap v181: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:10.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:10 smithi171 ceph-mon[33189]: pgmap v181: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 22 KiB/s wr, 6 op/s 2024-04-25T17:16:12.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:12 smithi096 ceph-mon[29131]: pgmap v182: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:12 smithi097 ceph-mon[32602]: pgmap v182: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:12.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:12 smithi171 ceph-mon[33189]: pgmap v182: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-04-25T17:16:14.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:14 smithi097 ceph-mon[32602]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T17:16:14.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:14 smithi171 ceph-mon[33189]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T17:16:14.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:14 smithi096 ceph-mon[29131]: pgmap v183: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T17:16:16.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:16 smithi171 ceph-mon[33189]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T17:16:16.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:16 smithi096 ceph-mon[29131]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T17:16:16.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:16 smithi097 ceph-mon[32602]: pgmap v184: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-25T17:16:18.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:18 smithi171 ceph-mon[33189]: pgmap v185: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T17:16:18.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:18 smithi096 ceph-mon[29131]: pgmap v185: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T17:16:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:18 smithi097 ceph-mon[32602]: pgmap v185: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T17:16:20.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:20 smithi171 ceph-mon[33189]: pgmap v186: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T17:16:20.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:20 smithi096 ceph-mon[29131]: pgmap v186: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T17:16:20.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:20 smithi097 ceph-mon[32602]: pgmap v186: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T17:16:22.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:22 smithi171 ceph-mon[33189]: pgmap v187: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T17:16:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:22 smithi096 ceph-mon[29131]: pgmap v187: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T17:16:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:22 smithi097 ceph-mon[32602]: pgmap v187: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T17:16:24.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:24 smithi171 ceph-mon[33189]: pgmap v188: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:24.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:24 smithi096 ceph-mon[29131]: pgmap v188: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:24 smithi097 ceph-mon[32602]: pgmap v188: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:25.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:25 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:16:25.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:25 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:16:25.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:25 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:16:26.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:26 smithi171 ceph-mon[33189]: pgmap v189: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:26.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:16:26.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:16:26.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:16:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:26 smithi096 ceph-mon[29131]: pgmap v189: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:16:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:16:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:16:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:26 smithi097 ceph-mon[32602]: pgmap v189: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:16:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:16:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:16:28.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:28 smithi171 ceph-mon[33189]: pgmap v190: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:28.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:28 smithi096 ceph-mon[29131]: pgmap v190: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:28 smithi097 ceph-mon[32602]: pgmap v190: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:30.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:30 smithi171 ceph-mon[33189]: pgmap v191: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:30.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:30 smithi096 ceph-mon[29131]: pgmap v191: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:30.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:30 smithi097 ceph-mon[32602]: pgmap v191: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:32.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:32 smithi171 ceph-mon[33189]: pgmap v192: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:32.786 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:32 smithi097 ceph-mon[32602]: pgmap v192: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:32.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:32 smithi096 ceph-mon[29131]: pgmap v192: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:34.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:34 smithi171 ceph-mon[33189]: pgmap v193: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:34.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:34 smithi096 ceph-mon[29131]: pgmap v193: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:34.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:34 smithi097 ceph-mon[32602]: pgmap v193: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:36.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:36 smithi171 ceph-mon[33189]: pgmap v194: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:36.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:36 smithi096 ceph-mon[29131]: pgmap v194: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:36.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:36 smithi097 ceph-mon[32602]: pgmap v194: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:38.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:38 smithi171 ceph-mon[33189]: pgmap v195: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:38.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:38 smithi096 ceph-mon[29131]: pgmap v195: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:38.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:38 smithi097 ceph-mon[32602]: pgmap v195: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:40.780 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:40 smithi171 ceph-mon[33189]: pgmap v196: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:40.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:40 smithi096 ceph-mon[29131]: pgmap v196: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:40.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:40 smithi097 ceph-mon[32602]: pgmap v196: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:42.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:42 smithi171 ceph-mon[33189]: pgmap v197: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:42 smithi096 ceph-mon[29131]: pgmap v197: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:42 smithi097 ceph-mon[32602]: pgmap v197: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:44.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:44 smithi096 ceph-mon[29131]: pgmap v198: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:44.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:44 smithi097 ceph-mon[32602]: pgmap v198: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:45.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:44 smithi171 ceph-mon[33189]: pgmap v198: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:46.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:46 smithi096 ceph-mon[29131]: pgmap v199: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:46 smithi097 ceph-mon[32602]: pgmap v199: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:47.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:46 smithi171 ceph-mon[33189]: pgmap v199: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:48.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:48 smithi096 ceph-mon[29131]: pgmap v200: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:48.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:48 smithi097 ceph-mon[32602]: pgmap v200: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:49.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:48 smithi171 ceph-mon[33189]: pgmap v200: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:50.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:50 smithi096 ceph-mon[29131]: pgmap v201: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:50.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:50 smithi097 ceph-mon[32602]: pgmap v201: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:51.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:50 smithi171 ceph-mon[33189]: pgmap v201: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:52.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:52 smithi096 ceph-mon[29131]: pgmap v202: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:52.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:52 smithi097 ceph-mon[32602]: pgmap v202: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:53.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:52 smithi171 ceph-mon[33189]: pgmap v202: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:54.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:54 smithi096 ceph-mon[29131]: pgmap v203: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:54.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:54 smithi097 ceph-mon[32602]: pgmap v203: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:55.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:54 smithi171 ceph-mon[33189]: pgmap v203: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:56.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:56 smithi096 ceph-mon[29131]: pgmap v204: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:56.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:56 smithi097 ceph-mon[32602]: pgmap v204: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:57.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:56 smithi171 ceph-mon[33189]: pgmap v204: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:16:58.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:16:58 smithi096 ceph-mon[29131]: pgmap v205: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:58.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:16:58 smithi097 ceph-mon[32602]: pgmap v205: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:16:59.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:16:58 smithi171 ceph-mon[33189]: pgmap v205: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:00.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:00 smithi096 ceph-mon[29131]: pgmap v206: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:00.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:00 smithi097 ceph-mon[32602]: pgmap v206: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:01.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:00 smithi171 ceph-mon[33189]: pgmap v206: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:02.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:02 smithi171 ceph-mon[33189]: pgmap v207: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:17:02.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:02 smithi096 ceph-mon[29131]: pgmap v207: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:17:02.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:02 smithi097 ceph-mon[32602]: pgmap v207: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:17:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:04 smithi096 ceph-mon[29131]: pgmap v208: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:04.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:04 smithi097 ceph-mon[32602]: pgmap v208: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:05.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:04 smithi171 ceph-mon[33189]: pgmap v208: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:06.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:06 smithi096 ceph-mon[29131]: pgmap v209: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:06.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:06 smithi097 ceph-mon[32602]: pgmap v209: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:07.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:06 smithi171 ceph-mon[33189]: pgmap v209: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:08.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:08 smithi096 ceph-mon[29131]: pgmap v210: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:17:08.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:08 smithi097 ceph-mon[32602]: pgmap v210: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:17:09.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:08 smithi171 ceph-mon[33189]: pgmap v210: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:17:10.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:10 smithi096 ceph-mon[29131]: pgmap v211: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:10.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:10 smithi097 ceph-mon[32602]: pgmap v211: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:11.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:10 smithi171 ceph-mon[33189]: pgmap v211: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:17:11.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:11 smithi096 ceph-mon[29131]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T17:17:11.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:11 smithi097 ceph-mon[32602]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T17:17:12.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:11 smithi171 ceph-mon[33189]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T17:17:12.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:12 smithi096 ceph-mon[29131]: pgmap v213: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:12.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:12 smithi097 ceph-mon[32602]: pgmap v213: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:13.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:12 smithi171 ceph-mon[33189]: pgmap v213: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:14.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:14 smithi096 ceph-mon[29131]: pgmap v214: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:14 smithi097 ceph-mon[32602]: pgmap v214: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:15.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:14 smithi171 ceph-mon[33189]: pgmap v214: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:16.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:16 smithi096 ceph-mon[29131]: pgmap v215: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:16.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:16 smithi097 ceph-mon[32602]: pgmap v215: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:17.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:16 smithi171 ceph-mon[33189]: pgmap v215: 97 pgs: 23 active+clean+snaptrim, 34 active+clean, 40 active+clean+snaptrim_wait; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:18.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:18 smithi096 ceph-mon[29131]: pgmap v216: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:18.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:18 smithi096 ceph-mon[29131]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T17:17:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:18 smithi097 ceph-mon[32602]: pgmap v216: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:18 smithi097 ceph-mon[32602]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T17:17:19.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:18 smithi171 ceph-mon[33189]: pgmap v216: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:17:19.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:18 smithi171 ceph-mon[33189]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T17:17:20.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:20 smithi096 ceph-mon[29131]: pgmap v218: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 245 B/s wr, 2 op/s 2024-04-25T17:17:20.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:20 smithi097 ceph-mon[32602]: pgmap v218: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 245 B/s wr, 2 op/s 2024-04-25T17:17:21.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:20 smithi171 ceph-mon[33189]: pgmap v218: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 245 B/s wr, 2 op/s 2024-04-25T17:17:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:22 smithi096 ceph-mon[29131]: pgmap v219: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:22 smithi097 ceph-mon[32602]: pgmap v219: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:23.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:22 smithi171 ceph-mon[33189]: pgmap v219: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:24.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:24 smithi096 ceph-mon[29131]: pgmap v220: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:24 smithi097 ceph-mon[32602]: pgmap v220: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:25.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:24 smithi171 ceph-mon[33189]: pgmap v220: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:26 smithi096 ceph-mon[29131]: pgmap v221: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:17:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:17:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:17:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:17:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:26 smithi097 ceph-mon[32602]: pgmap v221: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:17:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:17:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:17:26.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:17:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:26 smithi171 ceph-mon[33189]: pgmap v221: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:17:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:17:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:17:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:17:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:17:28.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:28 smithi096 ceph-mon[29131]: pgmap v222: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:17:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:28 smithi097 ceph-mon[32602]: pgmap v222: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:17:29.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:28 smithi171 ceph-mon[33189]: pgmap v222: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:17:30.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:30 smithi096 ceph-mon[29131]: pgmap v223: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-25T17:17:30.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:30 smithi097 ceph-mon[32602]: pgmap v223: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-25T17:17:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:30 smithi171 ceph-mon[33189]: pgmap v223: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-25T17:17:32.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:32 smithi096 ceph-mon[29131]: pgmap v224: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:32.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:32 smithi097 ceph-mon[32602]: pgmap v224: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:33.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:32 smithi171 ceph-mon[33189]: pgmap v224: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:34.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:34 smithi096 ceph-mon[29131]: pgmap v225: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:34.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:34 smithi097 ceph-mon[32602]: pgmap v225: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:34 smithi171 ceph-mon[33189]: pgmap v225: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:36.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:36 smithi096 ceph-mon[29131]: pgmap v226: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:36.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:36 smithi097 ceph-mon[32602]: pgmap v226: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:37.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:36 smithi171 ceph-mon[33189]: pgmap v226: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:38.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:38 smithi096 ceph-mon[29131]: pgmap v227: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:38.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:38 smithi097 ceph-mon[32602]: pgmap v227: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:39.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:38 smithi171 ceph-mon[33189]: pgmap v227: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:40.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:40 smithi096 ceph-mon[29131]: pgmap v228: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:40.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:40 smithi097 ceph-mon[32602]: pgmap v228: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:41.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:40 smithi171 ceph-mon[33189]: pgmap v228: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:42 smithi096 ceph-mon[29131]: pgmap v229: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:42 smithi097 ceph-mon[32602]: pgmap v229: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:42 smithi171 ceph-mon[33189]: pgmap v229: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:44.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:44 smithi096 ceph-mon[29131]: pgmap v230: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:44.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:44 smithi097 ceph-mon[32602]: pgmap v230: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:45.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:44 smithi171 ceph-mon[33189]: pgmap v230: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:46.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:46 smithi096 ceph-mon[29131]: pgmap v231: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:46 smithi097 ceph-mon[32602]: pgmap v231: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:47.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:46 smithi171 ceph-mon[33189]: pgmap v231: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:48.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:48 smithi096 ceph-mon[29131]: pgmap v232: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:48.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:48 smithi097 ceph-mon[32602]: pgmap v232: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:49.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:48 smithi171 ceph-mon[33189]: pgmap v232: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:50.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:50 smithi096 ceph-mon[29131]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:50.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:50 smithi097 ceph-mon[32602]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:51.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:50 smithi171 ceph-mon[33189]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:52.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:52 smithi096 ceph-mon[29131]: pgmap v234: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:52.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:52 smithi097 ceph-mon[32602]: pgmap v234: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:53.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:52 smithi171 ceph-mon[33189]: pgmap v234: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:54.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:54 smithi096 ceph-mon[29131]: pgmap v235: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:54.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:54 smithi097 ceph-mon[32602]: pgmap v235: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:55.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:54 smithi171 ceph-mon[33189]: pgmap v235: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:56.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:56 smithi096 ceph-mon[29131]: pgmap v236: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:56.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:56 smithi097 ceph-mon[32602]: pgmap v236: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:57.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:56 smithi171 ceph-mon[33189]: pgmap v236: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:17:58.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:17:58 smithi096 ceph-mon[29131]: pgmap v237: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:58.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:17:58 smithi097 ceph-mon[32602]: pgmap v237: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:17:59.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:17:58 smithi171 ceph-mon[33189]: pgmap v237: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:00.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:00 smithi096 ceph-mon[29131]: pgmap v238: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:00.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:00 smithi097 ceph-mon[32602]: pgmap v238: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:01.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:00 smithi171 ceph-mon[33189]: pgmap v238: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:02.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:02 smithi096 ceph-mon[29131]: pgmap v239: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:18:02.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:02 smithi097 ceph-mon[32602]: pgmap v239: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:18:03.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:02 smithi171 ceph-mon[33189]: pgmap v239: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:18:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:04 smithi096 ceph-mon[29131]: pgmap v240: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:04.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:04 smithi097 ceph-mon[32602]: pgmap v240: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:05.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:04 smithi171 ceph-mon[33189]: pgmap v240: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:06.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:06 smithi096 ceph-mon[29131]: pgmap v241: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:06.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:06 smithi097 ceph-mon[32602]: pgmap v241: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:07.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:06 smithi171 ceph-mon[33189]: pgmap v241: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:08.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:08 smithi096 ceph-mon[29131]: pgmap v242: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:18:08.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:08 smithi097 ceph-mon[32602]: pgmap v242: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:18:09.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:08 smithi171 ceph-mon[33189]: pgmap v242: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:18:10.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:10 smithi097 ceph-mon[32602]: pgmap v243: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:11.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:10 smithi171 ceph-mon[33189]: pgmap v243: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:11.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:10 smithi096 ceph-mon[29131]: pgmap v243: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 45 KiB/s wr, 8 op/s 2024-04-25T17:18:11.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:11 smithi097 ceph-mon[32602]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T17:18:12.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:11 smithi171 ceph-mon[33189]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T17:18:12.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:11 smithi096 ceph-mon[29131]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T17:18:12.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:12 smithi097 ceph-mon[32602]: pgmap v245: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:13.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:12 smithi171 ceph-mon[33189]: pgmap v245: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:13.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:12 smithi096 ceph-mon[29131]: pgmap v245: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:14 smithi097 ceph-mon[32602]: pgmap v246: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:15.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:14 smithi171 ceph-mon[33189]: pgmap v246: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:15.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:14 smithi096 ceph-mon[29131]: pgmap v246: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:15.189 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:18:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:18:14.943+0000 7f46eb6de700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-25T17:18:16.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:16 smithi097 ceph-mon[32602]: pgmap v247: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:17.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:16 smithi171 ceph-mon[33189]: pgmap v247: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:17.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:16 smithi096 ceph-mon[29131]: pgmap v247: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:18 smithi097 ceph-mon[32602]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:18 smithi097 ceph-mon[32602]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T17:18:19.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:18 smithi171 ceph-mon[33189]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:19.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:18 smithi171 ceph-mon[33189]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T17:18:19.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:18 smithi096 ceph-mon[29131]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:18:19.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:18 smithi096 ceph-mon[29131]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T17:18:20.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:20 smithi097 ceph-mon[32602]: pgmap v250: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-25T17:18:21.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:20 smithi171 ceph-mon[33189]: pgmap v250: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-25T17:18:21.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:20 smithi096 ceph-mon[29131]: pgmap v250: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-25T17:18:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:22 smithi097 ceph-mon[32602]: pgmap v251: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:23.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:22 smithi171 ceph-mon[33189]: pgmap v251: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:23.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:22 smithi096 ceph-mon[29131]: pgmap v251: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:24 smithi097 ceph-mon[32602]: pgmap v252: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:25.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:24 smithi171 ceph-mon[33189]: pgmap v252: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:25.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:24 smithi096 ceph-mon[29131]: pgmap v252: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:26 smithi097 ceph-mon[32602]: pgmap v253: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:26 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:18:27.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:26 smithi171 ceph-mon[33189]: pgmap v253: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:26 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:18:27.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:26 smithi096 ceph-mon[29131]: pgmap v253: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:18:27.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:26 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:18:28.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:27 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:18:28.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:27 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:18:28.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:27 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:18:28.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:27 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:18:28.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:27 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:18:28.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:27 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:18:28.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:27 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:18:28.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:27 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:18:28.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:27 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:18:29.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:28 smithi171 ceph-mon[33189]: pgmap v254: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:18:29.189 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:28 smithi096 ceph-mon[29131]: pgmap v254: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:18:29.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:28 smithi097 ceph-mon[32602]: pgmap v254: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:18:30.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:29 smithi171 ceph-mon[33189]: pgmap v255: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-25T17:18:30.090 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:29 smithi096 ceph-mon[29131]: pgmap v255: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-25T17:18:30.234 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:29 smithi097 ceph-mon[32602]: pgmap v255: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-25T17:18:32.280 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:31 smithi171 ceph-mon[33189]: pgmap v256: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:32.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:31 smithi096 ceph-mon[29131]: pgmap v256: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:32.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:31 smithi097 ceph-mon[32602]: pgmap v256: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:34.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:33 smithi171 ceph-mon[33189]: pgmap v257: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:34.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:33 smithi096 ceph-mon[29131]: pgmap v257: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:34.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:33 smithi097 ceph-mon[32602]: pgmap v257: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:36.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:36 smithi171 ceph-mon[33189]: pgmap v258: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:36.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:36 smithi096 ceph-mon[29131]: pgmap v258: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:36.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:36 smithi097 ceph-mon[32602]: pgmap v258: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:38.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:38 smithi171 ceph-mon[33189]: pgmap v259: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:38.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:38 smithi096 ceph-mon[29131]: pgmap v259: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:38.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:38 smithi097 ceph-mon[32602]: pgmap v259: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:40.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:40 smithi171 ceph-mon[33189]: pgmap v260: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:40.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:40 smithi096 ceph-mon[29131]: pgmap v260: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:40.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:40 smithi097 ceph-mon[32602]: pgmap v260: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:42.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:42 smithi171 ceph-mon[33189]: pgmap v261: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:42.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:42 smithi096 ceph-mon[29131]: pgmap v261: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:42.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:42 smithi097 ceph-mon[32602]: pgmap v261: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:44.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:44 smithi171 ceph-mon[33189]: pgmap v262: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:44.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:44 smithi096 ceph-mon[29131]: pgmap v262: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:44.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:44 smithi097 ceph-mon[32602]: pgmap v262: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:46.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:46 smithi171 ceph-mon[33189]: pgmap v263: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:46.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:46 smithi096 ceph-mon[29131]: pgmap v263: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:46.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:46 smithi097 ceph-mon[32602]: pgmap v263: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:48.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:48 smithi171 ceph-mon[33189]: pgmap v264: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:48.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:48 smithi096 ceph-mon[29131]: pgmap v264: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:48.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:48 smithi097 ceph-mon[32602]: pgmap v264: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:50.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:50 smithi171 ceph-mon[33189]: pgmap v265: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:50.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:50 smithi096 ceph-mon[29131]: pgmap v265: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:50.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:50 smithi097 ceph-mon[32602]: pgmap v265: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:52.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:52 smithi171 ceph-mon[33189]: pgmap v266: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:52 smithi096 ceph-mon[29131]: pgmap v266: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:52.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:52 smithi097 ceph-mon[32602]: pgmap v266: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:54.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:54 smithi096 ceph-mon[29131]: pgmap v267: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:54.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:54 smithi097 ceph-mon[32602]: pgmap v267: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:54.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:54 smithi171 ceph-mon[33189]: pgmap v267: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:56.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:56 smithi096 ceph-mon[29131]: pgmap v268: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:56 smithi097 ceph-mon[32602]: pgmap v268: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:56.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:56 smithi171 ceph-mon[33189]: pgmap v268: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:18:58.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:18:58 smithi096 ceph-mon[29131]: pgmap v269: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:58.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:18:58 smithi097 ceph-mon[32602]: pgmap v269: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:18:58.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:18:58 smithi171 ceph-mon[33189]: pgmap v269: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:00.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:00 smithi096 ceph-mon[29131]: pgmap v270: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:00.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:00 smithi097 ceph-mon[32602]: pgmap v270: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:00 smithi171 ceph-mon[33189]: pgmap v270: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:02.333 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:02 smithi171 ceph-mon[33189]: pgmap v271: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:19:02.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:02 smithi096 ceph-mon[29131]: pgmap v271: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:19:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:02 smithi097 ceph-mon[32602]: pgmap v271: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:19:04.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:04 smithi096 ceph-mon[29131]: pgmap v272: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:04 smithi097 ceph-mon[32602]: pgmap v272: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:04.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:04 smithi171 ceph-mon[33189]: pgmap v272: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:06.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:06 smithi096 ceph-mon[29131]: pgmap v273: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:06.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:06 smithi097 ceph-mon[32602]: pgmap v273: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:06.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:06 smithi171 ceph-mon[33189]: pgmap v273: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:08.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:08 smithi096 ceph-mon[29131]: pgmap v274: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:19:08.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:08 smithi097 ceph-mon[32602]: pgmap v274: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:19:08.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:08 smithi171 ceph-mon[33189]: pgmap v274: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:19:10.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:10 smithi096 ceph-mon[29131]: pgmap v275: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:10.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:10 smithi097 ceph-mon[32602]: pgmap v275: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:10.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:10 smithi171 ceph-mon[33189]: pgmap v275: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:19:12.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:12 smithi096 ceph-mon[29131]: pgmap v276: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:19:12.469 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:12 smithi096 ceph-mon[29131]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T17:19:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:12 smithi097 ceph-mon[32602]: pgmap v276: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:19:12.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:12 smithi097 ceph-mon[32602]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T17:19:12.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:12 smithi171 ceph-mon[33189]: pgmap v276: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 45 KiB/s wr, 9 op/s 2024-04-25T17:19:12.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:12 smithi171 ceph-mon[33189]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T17:19:14.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:14 smithi096 ceph-mon[29131]: pgmap v278: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:14.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:14 smithi097 ceph-mon[32602]: pgmap v278: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:14.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:14 smithi171 ceph-mon[33189]: pgmap v278: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:16.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:16 smithi096 ceph-mon[29131]: pgmap v279: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:16.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:16 smithi097 ceph-mon[32602]: pgmap v279: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:16.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:16 smithi171 ceph-mon[33189]: pgmap v279: 97 pgs: 97 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:18.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:18 smithi096 ceph-mon[29131]: pgmap v280: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:18.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:18 smithi097 ceph-mon[32602]: pgmap v280: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:18.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:18 smithi171 ceph-mon[33189]: pgmap v280: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:19:19.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:19 smithi096 ceph-mon[29131]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T17:19:19.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:19 smithi097 ceph-mon[32602]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T17:19:20.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:19 smithi171 ceph-mon[33189]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T17:19:20.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:20 smithi096 ceph-mon[29131]: pgmap v282: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T17:19:20.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:20 smithi097 ceph-mon[32602]: pgmap v282: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T17:19:21.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:20 smithi171 ceph-mon[33189]: pgmap v282: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T17:19:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:22 smithi096 ceph-mon[29131]: pgmap v283: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T17:19:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:22 smithi097 ceph-mon[32602]: pgmap v283: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T17:19:23.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:22 smithi171 ceph-mon[33189]: pgmap v283: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T17:19:24.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:24 smithi096 ceph-mon[29131]: pgmap v284: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:19:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:24 smithi097 ceph-mon[32602]: pgmap v284: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:19:25.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:24 smithi171 ceph-mon[33189]: pgmap v284: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:19:26.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:26 smithi096 ceph-mon[29131]: pgmap v285: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:19:26.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:26 smithi097 ceph-mon[32602]: pgmap v285: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:19:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:26 smithi171 ceph-mon[33189]: pgmap v285: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T17:19:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:27 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:19:27.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:27 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:19:28.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:27 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:19:28.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:28 smithi096 ceph-mon[29131]: pgmap v286: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:19:28.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:28 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:19:28.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:28 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:19:28.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:28 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:19:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:28 smithi097 ceph-mon[32602]: pgmap v286: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:19:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:28 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:19:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:28 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:19:28.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:28 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:19:29.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:28 smithi171 ceph-mon[33189]: pgmap v286: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:19:29.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:28 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:19:29.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:28 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:19:29.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:28 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:19:30.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:30 smithi096 ceph-mon[29131]: pgmap v287: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-25T17:19:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:30 smithi097 ceph-mon[32602]: pgmap v287: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-25T17:19:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:30 smithi171 ceph-mon[33189]: pgmap v287: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-25T17:19:32.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:32 smithi171 ceph-mon[33189]: pgmap v288: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:32.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:32 smithi096 ceph-mon[29131]: pgmap v288: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:32.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:32 smithi097 ceph-mon[32602]: pgmap v288: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:34.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:34 smithi096 ceph-mon[29131]: pgmap v289: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:34.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:34 smithi097 ceph-mon[32602]: pgmap v289: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:35.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:34 smithi171 ceph-mon[33189]: pgmap v289: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:36.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:36 smithi096 ceph-mon[29131]: pgmap v290: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:36.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:36 smithi097 ceph-mon[32602]: pgmap v290: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:37.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:36 smithi171 ceph-mon[33189]: pgmap v290: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:38.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:38 smithi096 ceph-mon[29131]: pgmap v291: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:38.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:38 smithi097 ceph-mon[32602]: pgmap v291: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:39.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:38 smithi171 ceph-mon[33189]: pgmap v291: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:40.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:40 smithi096 ceph-mon[29131]: pgmap v292: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:40.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:40 smithi097 ceph-mon[32602]: pgmap v292: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:41.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:40 smithi171 ceph-mon[33189]: pgmap v292: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:42.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:42 smithi096 ceph-mon[29131]: pgmap v293: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:42.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:42 smithi097 ceph-mon[32602]: pgmap v293: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:43.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:42 smithi171 ceph-mon[33189]: pgmap v293: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:44.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:44 smithi096 ceph-mon[29131]: pgmap v294: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:44.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:44 smithi097 ceph-mon[32602]: pgmap v294: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:45.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:44 smithi171 ceph-mon[33189]: pgmap v294: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:46.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:46 smithi096 ceph-mon[29131]: pgmap v295: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:46.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:46 smithi097 ceph-mon[32602]: pgmap v295: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:47.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:46 smithi171 ceph-mon[33189]: pgmap v295: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:48.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:48 smithi096 ceph-mon[29131]: pgmap v296: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:48.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:48 smithi097 ceph-mon[32602]: pgmap v296: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:49.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:48 smithi171 ceph-mon[33189]: pgmap v296: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:50.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:50 smithi096 ceph-mon[29131]: pgmap v297: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:50.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:50 smithi097 ceph-mon[32602]: pgmap v297: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:51.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:50 smithi171 ceph-mon[33189]: pgmap v297: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:52.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:52 smithi096 ceph-mon[29131]: pgmap v298: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:52.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:52 smithi097 ceph-mon[32602]: pgmap v298: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:53.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:52 smithi171 ceph-mon[33189]: pgmap v298: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:54.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:54 smithi096 ceph-mon[29131]: pgmap v299: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:54.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:54 smithi097 ceph-mon[32602]: pgmap v299: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:55.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:54 smithi171 ceph-mon[33189]: pgmap v299: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:56.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:56 smithi096 ceph-mon[29131]: pgmap v300: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:56.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:56 smithi097 ceph-mon[32602]: pgmap v300: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:57.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:56 smithi171 ceph-mon[33189]: pgmap v300: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:19:58.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:19:58 smithi096 ceph-mon[29131]: pgmap v301: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:58.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:19:58 smithi097 ceph-mon[32602]: pgmap v301: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:19:59.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:19:58 smithi171 ceph-mon[33189]: pgmap v301: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:00.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:00 smithi096 ceph-mon[29131]: pgmap v302: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:00.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:00 smithi096 ceph-mon[29131]: overall HEALTH_OK 2024-04-25T17:20:00.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:00 smithi097 ceph-mon[32602]: pgmap v302: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:00.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:00 smithi097 ceph-mon[32602]: overall HEALTH_OK 2024-04-25T17:20:01.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:00 smithi171 ceph-mon[33189]: pgmap v302: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:01.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:00 smithi171 ceph-mon[33189]: overall HEALTH_OK 2024-04-25T17:20:02.802 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:02 smithi171 ceph-mon[33189]: pgmap v303: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:20:02.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:02 smithi096 ceph-mon[29131]: pgmap v303: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:20:02.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:02 smithi097 ceph-mon[32602]: pgmap v303: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:20:04.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:04 smithi096 ceph-mon[29131]: pgmap v304: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:04.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:04 smithi097 ceph-mon[32602]: pgmap v304: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:05.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:04 smithi171 ceph-mon[33189]: pgmap v304: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:06.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:06 smithi096 ceph-mon[29131]: pgmap v305: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:06.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:06 smithi097 ceph-mon[32602]: pgmap v305: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:07.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:06 smithi171 ceph-mon[33189]: pgmap v305: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:08.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:08 smithi096 ceph-mon[29131]: pgmap v306: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:20:08.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:08 smithi097 ceph-mon[32602]: pgmap v306: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:20:09.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:08 smithi171 ceph-mon[33189]: pgmap v306: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-04-25T17:20:10.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:10 smithi096 ceph-mon[29131]: pgmap v307: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:10.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:10 smithi097 ceph-mon[32602]: pgmap v307: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:11.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:10 smithi171 ceph-mon[33189]: pgmap v307: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-04-25T17:20:11.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:11 smithi096 ceph-mon[29131]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T17:20:11.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:11 smithi097 ceph-mon[32602]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T17:20:12.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:11 smithi171 ceph-mon[33189]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T17:20:12.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:12 smithi096 ceph-mon[29131]: pgmap v309: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:12.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:12 smithi097 ceph-mon[32602]: pgmap v309: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:13.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:12 smithi171 ceph-mon[33189]: pgmap v309: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:14.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:14 smithi096 ceph-mon[29131]: pgmap v310: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:14.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:14 smithi097 ceph-mon[32602]: pgmap v310: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:15.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:14 smithi171 ceph-mon[33189]: pgmap v310: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:15.439 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:20:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:20:14.977+0000 7f46eb6de700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-25T17:20:16.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:16 smithi096 ceph-mon[29131]: pgmap v311: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:16.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:16 smithi097 ceph-mon[32602]: pgmap v311: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:17.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:16 smithi171 ceph-mon[33189]: pgmap v311: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 75 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:17.903 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 -- 172.21.15.96:0/3963845870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8414103a80 msgr2=0x7f8414103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 --2- 172.21.15.96:0/3963845870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8414103a80 0x7f8414103e90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8408009900 tx=0x7f840802a410 comp rx=0 tx=0).stop 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 -- 172.21.15.96:0/3963845870 shutdown_connections 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 --2- 172.21.15.96:0/3963845870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f841410e310 0x7f8414110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 --2- 172.21.15.96:0/3963845870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8414103a80 0x7f8414103e90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 --2- 172.21.15.96:0/3963845870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84141099b0 0x7f8414103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 -- 172.21.15.96:0/3963845870 >> 172.21.15.96:0/3963845870 conn(0x7f8414079080 msgr2=0x7f84140794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 -- 172.21.15.96:0/3963845870 shutdown_connections 2024-04-25T17:20:18.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 -- 172.21.15.96:0/3963845870 wait complete. 2024-04-25T17:20:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.026+0000 7f841359e640 1 Processor -- start 2024-04-25T17:20:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841359e640 1 -- start start 2024-04-25T17:20:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841359e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8414103a80 0x7f841419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84141099b0 0x7f841419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f841410e310 0x7f841419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841359e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8414113b70 con 0x7f84141099b0 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841359e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84141139f0 con 0x7f841410e310 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841359e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8414113cf0 con 0x7f8414103a80 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84141099b0 0x7f841419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84141099b0 0x7f841419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59666/0 (socket says 172.21.15.96:59666) 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 -- 172.21.15.96:0/350599598 learned_addr learned my addr 172.21.15.96:0/350599598 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8412d9d640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f841410e310 0x7f841419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f841259c640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8414103a80 0x7f841419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 -- 172.21.15.96:0/350599598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8414103a80 msgr2=0x7f841419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8414103a80 0x7f841419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 -- 172.21.15.96:0/350599598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f841410e310 msgr2=0x7f841419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f841410e310 0x7f841419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 -- 172.21.15.96:0/350599598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841419f370 con 0x7f84141099b0 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8412d9d640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f841410e310 0x7f841419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.027+0000 7f8411d9b640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84141099b0 0x7f841419aaf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8408032fd0 tx=0x7f8408004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.028+0000 7f84037fe640 1 -- 172.21.15.96:0/350599598 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840803f030 con 0x7f84141099b0 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.028+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841419f600 con 0x7f84141099b0 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.028+0000 7f84037fe640 1 -- 172.21.15.96:0/350599598 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8408004490 con 0x7f84141099b0 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.028+0000 7f84037fe640 1 -- 172.21.15.96:0/350599598 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408035070 con 0x7f84141099b0 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.028+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841419faf0 con 0x7f84141099b0 2024-04-25T17:20:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.029+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8414050740 con 0x7f84141099b0 2024-04-25T17:20:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.030+0000 7f84037fe640 1 -- 172.21.15.96:0/350599598 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8408042030 con 0x7f84141099b0 2024-04-25T17:20:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.031+0000 7f84037fe640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83fc0afcd0 0x7f83fc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.031+0000 7f841259c640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83fc0afcd0 0x7f83fc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.031+0000 7f84037fe640 1 -- 172.21.15.96:0/350599598 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f84080ffa90 con 0x7f84141099b0 2024-04-25T17:20:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.031+0000 7f841259c640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83fc0afcd0 0x7f83fc0b21e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8404005fd0 tx=0x7f8404005850 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:18.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.035+0000 7f84037fe640 1 -- 172.21.15.96:0/350599598 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84080c8290 con 0x7f84141099b0 2024-04-25T17:20:18.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.232+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 --> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8414107ed0 con 0x7f83fc0afcd0 2024-04-25T17:20:18.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.242+0000 7f84037fe640 1 -- 172.21.15.96:0/350599598 <== mgr.15261 v2:172.21.15.96:6800/1594342167 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f8414107ed0 con 0x7f83fc0afcd0 2024-04-25T17:20:18.243 INFO:teuthology.orchestra.run.smithi096.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T17:10:15", "first": "2024-04-25T17:11:00", "last": "2024-04-25T17:20:00", "last_pruned": "2024-04-25T17:20:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83fc0afcd0 msgr2=0x7f83fc0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83fc0afcd0 0x7f83fc0b21e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8404005fd0 tx=0x7f8404005850 comp rx=0 tx=0).stop 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84141099b0 msgr2=0x7f841419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84141099b0 0x7f841419aaf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8408032fd0 tx=0x7f8408004270 comp rx=0 tx=0).stop 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 shutdown_connections 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83fc0afcd0 0x7f83fc0b21e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f841410e310 0x7f841419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84141099b0 0x7f841419aaf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 --2- 172.21.15.96:0/350599598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8414103a80 0x7f841419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 >> 172.21.15.96:0/350599598 conn(0x7f8414079080 msgr2=0x7f8414105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:18.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 shutdown_connections 2024-04-25T17:20:18.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.244+0000 7f841359e640 1 -- 172.21.15.96:0/350599598 wait complete. 2024-04-25T17:20:18.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/4216022323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8102090 msgr2=0x7f8dd81024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/4216022323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8102090 0x7f8dd81024a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009900 tx=0x7f8dc802a410 comp rx=0 tx=0).stop 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/4216022323 shutdown_connections 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/4216022323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd810e590 0x7f8dd81109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/4216022323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8102090 0x7f8dd81024a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/4216022323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8dd8109be0 0x7f8dd8101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/4216022323 >> 172.21.15.96:0/4216022323 conn(0x7f8dd80fd8a0 msgr2=0x7f8dd80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/4216022323 shutdown_connections 2024-04-25T17:20:18.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.364+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/4216022323 wait complete. 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 Processor -- start 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 -- start start 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd8102090 0x7f8dd819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8109be0 0x7f8dd819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8dd810e590 0x7f8dd819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8113d50 con 0x7f8dd8109be0 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8113bd0 con 0x7f8dd8102090 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddfa0c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd8113ed0 con 0x7f8dd810e590 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8109be0 0x7f8dd819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8109be0 0x7f8dd819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59692/0 (socket says 172.21.15.96:59692) 2024-04-25T17:20:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 -- 172.21.15.96:0/2498329923 learned_addr learned my addr 172.21.15.96:0/2498329923 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddd7a8640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd8102090 0x7f8dd819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 -- 172.21.15.96:0/2498329923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8dd810e590 msgr2=0x7f8dd819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8dddfa9640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8dd810e590 0x7f8dd819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8dd810e590 0x7f8dd819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 -- 172.21.15.96:0/2498329923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd8102090 msgr2=0x7f8dd819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd8102090 0x7f8dd819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddcfa7640 1 -- 172.21.15.96:0/2498329923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd819f510 con 0x7f8dd8109be0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8dddfa9640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8dd810e590 0x7f8dd819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.365+0000 7f8ddd7a8640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd8102090 0x7f8dd819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.366+0000 7f8ddcfa7640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8109be0 0x7f8dd819ac90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8004270 tx=0x7f8dc80042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.366+0000 7f8dce7fc640 1 -- 172.21.15.96:0/2498329923 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc803f030 con 0x7f8dd8109be0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.366+0000 7f8dce7fc640 1 -- 172.21.15.96:0/2498329923 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc8003cb0 con 0x7f8dd8109be0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.366+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd819f7a0 con 0x7f8dd8109be0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.366+0000 7f8dce7fc640 1 -- 172.21.15.96:0/2498329923 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8035070 con 0x7f8dd8109be0 2024-04-25T17:20:18.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.366+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd819fc90 con 0x7f8dd8109be0 2024-04-25T17:20:18.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.367+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd8050740 con 0x7f8dd8109be0 2024-04-25T17:20:18.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.368+0000 7f8dce7fc640 1 -- 172.21.15.96:0/2498329923 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8dc8042030 con 0x7f8dd8109be0 2024-04-25T17:20:18.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.369+0000 7f8dce7fc640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8dc40afc50 0x7f8dc40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.369+0000 7f8ddd7a8640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8dc40afc50 0x7f8dc40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.369+0000 7f8dce7fc640 1 -- 172.21.15.96:0/2498329923 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8586+0+0 (secure 0 0 0) 0x7f8dc80ffb40 con 0x7f8dd8109be0 2024-04-25T17:20:18.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.370+0000 7f8ddd7a8640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8dc40afc50 0x7f8dc40b2160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8dd806bda0 tx=0x7f8dd4007870 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:18.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.373+0000 7f8dce7fc640 1 -- 172.21.15.96:0/2498329923 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8dc80c83a0 con 0x7f8dd8109be0 2024-04-25T17:20:18.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.572+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 --> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8dd80fe3c0 con 0x7f8dc40afc50 2024-04-25T17:20:18.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.585+0000 7f8dce7fc640 1 -- 172.21.15.96:0/2498329923 <== mgr.15261 v2:172.21.15.96:6800/1594342167 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8dd80fe3c0 con 0x7f8dc40afc50 2024-04-25T17:20:18.586 INFO:teuthology.orchestra.run.smithi096.stdout:/ 1m 6m3h 2024-04-25T17:20:18.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8dc40afc50 msgr2=0x7f8dc40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8dc40afc50 0x7f8dc40b2160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8dd806bda0 tx=0x7f8dd4007870 comp rx=0 tx=0).stop 2024-04-25T17:20:18.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8109be0 msgr2=0x7f8dd819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8109be0 0x7f8dd819ac90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8004270 tx=0x7f8dc80042a0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 shutdown_connections 2024-04-25T17:20:18.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8dd810e590 0x7f8dd819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dd8109be0 0x7f8dd819ac90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f8dc40afc50 0x7f8dc40b2160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 --2- 172.21.15.96:0/2498329923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd8102090 0x7f8dd819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 >> 172.21.15.96:0/2498329923 conn(0x7f8dd80fd8a0 msgr2=0x7f8dd80fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:18.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 shutdown_connections 2024-04-25T17:20:18.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.587+0000 7f8ddfa0c640 1 -- 172.21.15.96:0/2498329923 wait complete. 2024-04-25T17:20:18.599 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T17:20:18.609 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:20:18.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 -- 172.21.15.96:0/1110240552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548108a20 msgr2=0x7fa5481025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1110240552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548108a20 0x7fa5481025b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa538009900 tx=0x7fa53802a410 comp rx=0 tx=0).stop 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 -- 172.21.15.96:0/1110240552 shutdown_connections 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1110240552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa54810d380 0x7fa54810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1110240552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548102af0 0x7fa548102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1110240552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548108a20 0x7fa5481025b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 -- 172.21.15.96:0/1110240552 >> 172.21.15.96:0/1110240552 conn(0x7fa548078110 msgr2=0x7fa548078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 -- 172.21.15.96:0/1110240552 shutdown_connections 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.717+0000 7fa54d97d640 1 -- 172.21.15.96:0/1110240552 wait complete. 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 Processor -- start 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 -- start start 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548102af0 0x7fa54819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548108a20 0x7fa54819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa54810d380 0x7fa54819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa548112be0 con 0x7fa548102af0 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa548112a60 con 0x7fa548108a20 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa54d97d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa548112d60 con 0x7fa54810d380 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548102af0 0x7fa54819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548102af0 0x7fa54819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59722/0 (socket says 172.21.15.96:59722) 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.718+0000 7fa546ffd640 1 -- 172.21.15.96:0/1066742963 learned_addr learned my addr 172.21.15.96:0/1066742963 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa5477fe640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa54810d380 0x7fa54819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa53e7fc640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548108a20 0x7fa54819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa546ffd640 1 -- 172.21.15.96:0/1066742963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa54810d380 msgr2=0x7fa54819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa546ffd640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa54810d380 0x7fa54819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa546ffd640 1 -- 172.21.15.96:0/1066742963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548108a20 msgr2=0x7fa54819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa546ffd640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548108a20 0x7fa54819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa546ffd640 1 -- 172.21.15.96:0/1066742963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54819f430 con 0x7fa548102af0 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa546ffd640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548102af0 0x7fa54819a650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa538004270 tx=0x7fa5380042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa54c97b640 1 -- 172.21.15.96:0/1066742963 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53803f030 con 0x7fa548102af0 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54819f6c0 con 0x7fa548102af0 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa54c97b640 1 -- 172.21.15.96:0/1066742963 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa538003cb0 con 0x7fa548102af0 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa54c97b640 1 -- 172.21.15.96:0/1066742963 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538035070 con 0x7fa548102af0 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.719+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54819fbb0 con 0x7fa548102af0 2024-04-25T17:20:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.720+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa548103cd0 con 0x7fa548102af0 2024-04-25T17:20:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.721+0000 7fa54c97b640 1 -- 172.21.15.96:0/1066742963 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fa538042030 con 0x7fa548102af0 2024-04-25T17:20:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.722+0000 7fa54c97b640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa5340afc60 0x7fa5340b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.722+0000 7fa53e7fc640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa5340afc60 0x7fa5340b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.722+0000 7fa54c97b640 1 -- 172.21.15.96:0/1066742963 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fa5380ffb10 con 0x7fa548102af0 2024-04-25T17:20:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.723+0000 7fa53e7fc640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa5340afc60 0x7fa5340b2170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5300058c0 tx=0x7fa530005850 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:18.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.726+0000 7fa54c97b640 1 -- 172.21.15.96:0/1066742963 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5380c83a0 con 0x7fa548102af0 2024-04-25T17:20:18.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.903+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5481a7120 con 0x7fa548102af0 2024-04-25T17:20:18.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.905+0000 7fa54c97b640 1 -- 172.21.15.96:0/1066742963 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23060 (secure 0 0 0) 0x7fa538003e60 con 0x7fa548102af0 2024-04-25T17:20:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:20:18.906 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:20:18.541083+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/559998201":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/1944193211":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/3419689135":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6801/3530143863":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6800/3530143863":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/328848234":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa5340afc60 msgr2=0x7fa5340b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa5340afc60 0x7fa5340b2170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5300058c0 tx=0x7fa530005850 comp rx=0 tx=0).stop 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548102af0 msgr2=0x7fa54819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548102af0 0x7fa54819a650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa538004270 tx=0x7fa5380042a0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 shutdown_connections 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa54810d380 0x7fa54819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fa5340afc60 0x7fa5340b2170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa548108a20 0x7fa54819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 --2- 172.21.15.96:0/1066742963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa548102af0 0x7fa54819a650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 >> 172.21.15.96:0/1066742963 conn(0x7fa548078110 msgr2=0x7fa548069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 shutdown_connections 2024-04-25T17:20:18.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:18.907+0000 7fa54d97d640 1 -- 172.21.15.96:0/1066742963 wait complete. 2024-04-25T17:20:18.917 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:18 smithi096 ceph-mon[29131]: pgmap v312: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:18.917 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:18 smithi096 ceph-mon[29131]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T17:20:18.919 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T17:08:09.266201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T17:08:10.452964+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:20:18.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:20:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:18 smithi097 ceph-mon[32602]: pgmap v312: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:18.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:18 smithi097 ceph-mon[32602]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T17:20:19.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:18 smithi171 ceph-mon[33189]: pgmap v312: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T17:20:19.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:18 smithi171 ceph-mon[33189]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 -- 172.21.15.96:0/2766117161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 msgr2=0x7fc008102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/2766117161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc008102460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009ca0 tx=0x7fbff802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 -- 172.21.15.96:0/2766117161 shutdown_connections 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/2766117161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00810e550 0x7fc008110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/2766117161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc008102460 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/2766117161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc008109ba0 0x7fc008101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 -- 172.21.15.96:0/2766117161 >> 172.21.15.96:0/2766117161 conn(0x7fc008077880 msgr2=0x7fc008077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 -- 172.21.15.96:0/2766117161 shutdown_connections 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 -- 172.21.15.96:0/2766117161 wait complete. 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.031+0000 7fc00f0ee640 1 Processor -- start 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 -- start start 2024-04-25T17:20:19.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc00819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc008109ba0 0x7fc00819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00810e550 0x7fc00819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc008113d10 con 0x7fc008102050 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc008113b90 con 0x7fc008109ba0 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc008113e90 con 0x7fc00810e550 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc00819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc00819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59742/0 (socket says 172.21.15.96:59742) 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 -- 172.21.15.96:0/237430898 learned_addr learned my addr 172.21.15.96:0/237430898 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 -- 172.21.15.96:0/237430898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00810e550 msgr2=0x7fc00819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00810e550 0x7fc00819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 -- 172.21.15.96:0/237430898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc008109ba0 msgr2=0x7fc00819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc008109ba0 0x7fc00819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 -- 172.21.15.96:0/237430898 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00819f5b0 con 0x7fc008102050 2024-04-25T17:20:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00ce8a640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc00819a750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbff0005d00 tx=0x7fbff000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc005ffb640 1 -- 172.21.15.96:0/237430898 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017070 con 0x7fc008102050 2024-04-25T17:20:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc005ffb640 1 -- 172.21.15.96:0/237430898 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff0009c30 con 0x7fc008102050 2024-04-25T17:20:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc005ffb640 1 -- 172.21.15.96:0/237430898 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff000f620 con 0x7fc008102050 2024-04-25T17:20:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.032+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00819f8a0 con 0x7fc008102050 2024-04-25T17:20:19.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.033+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00819fda0 con 0x7fc008102050 2024-04-25T17:20:19.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.034+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0080692e0 con 0x7fc008102050 2024-04-25T17:20:19.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.035+0000 7fc005ffb640 1 -- 172.21.15.96:0/237430898 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fbff001a070 con 0x7fc008102050 2024-04-25T17:20:19.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.035+0000 7fc005ffb640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fbff40afc60 0x7fbff40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.036+0000 7fc005ffb640 1 -- 172.21.15.96:0/237430898 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fbff00deb50 con 0x7fc008102050 2024-04-25T17:20:19.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.036+0000 7fc007fff640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fbff40afc60 0x7fbff40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.036+0000 7fc007fff640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fbff40afc60 0x7fbff40b2170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbff80094e0 tx=0x7fbff8000f40 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.040+0000 7fc005ffb640 1 -- 172.21.15.96:0/237430898 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc00819fda0 con 0x7fc008102050 2024-04-25T17:20:19.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.218+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc00802db50 con 0x7fc008102050 2024-04-25T17:20:19.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.219+0000 7fc005ffb640 1 -- 172.21.15.96:0/237430898 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fbff0009de0 con 0x7fc008102050 2024-04-25T17:20:19.220 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fbff40afc60 msgr2=0x7fbff40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fbff40afc60 0x7fbff40b2170 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbff80094e0 tx=0x7fbff8000f40 comp rx=0 tx=0).stop 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 msgr2=0x7fc00819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc00819a750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbff0005d00 tx=0x7fbff000dde0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 shutdown_connections 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00810e550 0x7fc00819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fbff40afc60 0x7fbff40b2170 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc008109ba0 0x7fc00819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 --2- 172.21.15.96:0/237430898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc008102050 0x7fc00819a750 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 >> 172.21.15.96:0/237430898 conn(0x7fc008077880 msgr2=0x7fc00810a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:19.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 shutdown_connections 2024-04-25T17:20:19.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.221+0000 7fc00f0ee640 1 -- 172.21.15.96:0/237430898 wait complete. 2024-04-25T17:20:19.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T17:20:19.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/39287349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4107450 msgr2=0x7f7cd410ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/39287349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4107450 0x7f7cd410ebf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00aa10 tx=0x7f7ccc02b720 comp rx=0 tx=0).stop 2024-04-25T17:20:19.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/39287349 shutdown_connections 2024-04-25T17:20:19.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/39287349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4107450 0x7f7cd410ebf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/39287349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd4106990 0x7f7cd4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/39287349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd4102400 0x7f7cd4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/39287349 >> 172.21.15.96:0/39287349 conn(0x7f7cd4077880 msgr2=0x7f7cd4077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/39287349 shutdown_connections 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.344+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/39287349 wait complete. 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 Processor -- start 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 -- start start 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd4102400 0x7f7cd419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4106990 0x7f7cd419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd4107450 0x7f7cd419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4111ae0 con 0x7f7cd4106990 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4111960 con 0x7f7cd4102400 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cdb7fe640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4111c60 con 0x7f7cd4107450 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4106990 0x7f7cd419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4106990 0x7f7cd419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59764/0 (socket says 172.21.15.96:59764) 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 -- 172.21.15.96:0/3051271924 learned_addr learned my addr 172.21.15.96:0/3051271924 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd959a640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd4102400 0x7f7cd419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd9d9b640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd4107450 0x7f7cd419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 -- 172.21.15.96:0/3051271924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd4107450 msgr2=0x7f7cd419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd4107450 0x7f7cd419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 -- 172.21.15.96:0/3051271924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd4102400 msgr2=0x7f7cd419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd4102400 0x7f7cd419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 -- 172.21.15.96:0/3051271924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd419f580 con 0x7f7cd4106990 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd959a640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd4102400 0x7f7cd419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd9d9b640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd4107450 0x7f7cd419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.345+0000 7f7cd8d99640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4106990 0x7f7cd419ad00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800d770 tx=0x7f7cc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.346+0000 7f7cd27fc640 1 -- 172.21.15.96:0/3051271924 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8017070 con 0x7f7cd4106990 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.346+0000 7f7cd27fc640 1 -- 172.21.15.96:0/3051271924 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc800de80 con 0x7f7cd4106990 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.346+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd419f870 con 0x7f7cd4106990 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.346+0000 7f7cd27fc640 1 -- 172.21.15.96:0/3051271924 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8004ce0 con 0x7f7cd4106990 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.346+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd419fd70 con 0x7f7cd4106990 2024-04-25T17:20:19.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.347+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd40692e0 con 0x7f7cd4106990 2024-04-25T17:20:19.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.348+0000 7f7cd27fc640 1 -- 172.21.15.96:0/3051271924 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7cc8004e80 con 0x7f7cd4106990 2024-04-25T17:20:19.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.348+0000 7f7cd27fc640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7cbc0afc60 0x7f7cbc0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.349+0000 7f7cd959a640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7cbc0afc60 0x7f7cbc0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.349+0000 7f7cd27fc640 1 -- 172.21.15.96:0/3051271924 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f7cc80dee30 con 0x7f7cd4106990 2024-04-25T17:20:19.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.349+0000 7f7cd959a640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7cbc0afc60 0x7f7cbc0b2170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4007e70 tx=0x7f7cc4007850 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.352+0000 7f7cd27fc640 1 -- 172.21.15.96:0/3051271924 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cc80a76c0 con 0x7f7cd4106990 2024-04-25T17:20:19.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.529+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7cd410c280 con 0x7f7cd4106990 2024-04-25T17:20:19.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.530+0000 7f7cd27fc640 1 -- 172.21.15.96:0/3051271924 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f7cc80a6fc0 con 0x7f7cd4106990 2024-04-25T17:20:19.531 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7cbc0afc60 msgr2=0x7f7cbc0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7cbc0afc60 0x7f7cbc0b2170 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4007e70 tx=0x7f7cc4007850 comp rx=0 tx=0).stop 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4106990 msgr2=0x7f7cd419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4106990 0x7f7cd419ad00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800d770 tx=0x7f7cc800dc30 comp rx=0 tx=0).stop 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 shutdown_connections 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7cd4107450 0x7f7cd419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7cd4106990 0x7f7cd419ad00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7cbc0afc60 0x7f7cbc0b2170 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 --2- 172.21.15.96:0/3051271924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7cd4102400 0x7f7cd419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 >> 172.21.15.96:0/3051271924 conn(0x7f7cd4077880 msgr2=0x7f7cd4105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 shutdown_connections 2024-04-25T17:20:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.532+0000 7f7cdb7fe640 1 -- 172.21.15.96:0/3051271924 wait complete. 2024-04-25T17:20:19.544 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T17:20:19.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 -- 172.21.15.96:0/3039537452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d8109ba0 msgr2=0x7f57d8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/3039537452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d8109ba0 0x7f57d8101c30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009900 tx=0x7f57c802a410 comp rx=0 tx=0).stop 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 -- 172.21.15.96:0/3039537452 shutdown_connections 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/3039537452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d81027e0 0x7f57d8110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/3039537452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57d810a040 0x7f57d8102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/3039537452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d8109ba0 0x7f57d8101c30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 -- 172.21.15.96:0/3039537452 >> 172.21.15.96:0/3039537452 conn(0x7f57d8077880 msgr2=0x7f57d8077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 -- 172.21.15.96:0/3039537452 shutdown_connections 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.654+0000 7f57ddaa7640 1 -- 172.21.15.96:0/3039537452 wait complete. 2024-04-25T17:20:19.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.655+0000 7f57ddaa7640 1 Processor -- start 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.655+0000 7f57ddaa7640 1 -- start start 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.655+0000 7f57ddaa7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57d81027e0 0x7f57d81a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.655+0000 7f57ddaa7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8109ba0 0x7f57d81a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.655+0000 7f57ddaa7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d810a040 0x7f57d819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.655+0000 7f57ddaa7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8113d80 con 0x7f57d810a040 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.655+0000 7f57ddaa7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8113c00 con 0x7f57d81027e0 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57ddaa7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8113f00 con 0x7f57d8109ba0 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d810a040 0x7f57d819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d810a040 0x7f57d819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59782/0 (socket says 172.21.15.96:59782) 2024-04-25T17:20:19.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 -- 172.21.15.96:0/448176426 learned_addr learned my addr 172.21.15.96:0/448176426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d77fe640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57d81027e0 0x7f57d81a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d6ffd640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8109ba0 0x7f57d81a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 -- 172.21.15.96:0/448176426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8109ba0 msgr2=0x7f57d81a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8109ba0 0x7f57d81a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 -- 172.21.15.96:0/448176426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57d81027e0 msgr2=0x7f57d81a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57d81027e0 0x7f57d81a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 -- 172.21.15.96:0/448176426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d819ad90 con 0x7f57d810a040 2024-04-25T17:20:19.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.656+0000 7f57d7fff640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d810a040 0x7f57d819a660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57d000eb40 tx=0x7f57d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.662+0000 7f57d4ff9640 1 -- 172.21.15.96:0/448176426 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0004b20 con 0x7f57d810a040 2024-04-25T17:20:19.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.662+0000 7f57d4ff9640 1 -- 172.21.15.96:0/448176426 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0004cc0 con 0x7f57d810a040 2024-04-25T17:20:19.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.662+0000 7f57d4ff9640 1 -- 172.21.15.96:0/448176426 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0010720 con 0x7f57d810a040 2024-04-25T17:20:19.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.662+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d819b020 con 0x7f57d810a040 2024-04-25T17:20:19.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.662+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81a7280 con 0x7f57d810a040 2024-04-25T17:20:19.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.663+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d8104f50 con 0x7f57d810a040 2024-04-25T17:20:19.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.664+0000 7f57d4ff9640 1 -- 172.21.15.96:0/448176426 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f57d0005560 con 0x7f57d810a040 2024-04-25T17:20:19.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.665+0000 7f57d4ff9640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f57c00afce0 0x7f57c00b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.665+0000 7f57d4ff9640 1 -- 172.21.15.96:0/448176426 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f57d00df780 con 0x7f57d810a040 2024-04-25T17:20:19.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.665+0000 7f57d77fe640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f57c00afce0 0x7f57c00b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.666+0000 7f57d77fe640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f57c00afce0 0x7f57c00b21f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57c8031440 tx=0x7f57c8009560 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.670+0000 7f57d4ff9640 1 -- 172.21.15.96:0/448176426 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57d00a8010 con 0x7f57d810a040 2024-04-25T17:20:19.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.848+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f57d806a160 con 0x7f57d810a040 2024-04-25T17:20:19.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.848+0000 7f57d4ff9640 1 -- 172.21.15.96:0/448176426 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f57d00a7910 con 0x7f57d810a040 2024-04-25T17:20:19.849 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T17:20:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f57c00afce0 msgr2=0x7f57c00b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f57c00afce0 0x7f57c00b21f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57c8031440 tx=0x7f57c8009560 comp rx=0 tx=0).stop 2024-04-25T17:20:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d810a040 msgr2=0x7f57d819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d810a040 0x7f57d819a660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f57d000eb40 tx=0x7f57d000c2d0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 shutdown_connections 2024-04-25T17:20:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57d810a040 0x7f57d819a660 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57d8109ba0 0x7f57d81a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f57c00afce0 0x7f57c00b21f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 --2- 172.21.15.96:0/448176426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57d81027e0 0x7f57d81a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 >> 172.21.15.96:0/448176426 conn(0x7f57d8077880 msgr2=0x7f57d8100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 shutdown_connections 2024-04-25T17:20:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.851+0000 7f57ddaa7640 1 -- 172.21.15.96:0/448176426 wait complete. 2024-04-25T17:20:19.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:19 smithi096 ceph-mon[29131]: from='client.15759 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:20:19.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:19 smithi096 ceph-mon[29131]: from='client.15765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:20:19.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:19 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1066742963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:19.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:19 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/237430898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:20:19.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:19 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3051271924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:20:19.862 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T17:20:19.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 -- 172.21.15.96:0/564361272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c104280 msgr2=0x7f7a5c110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 --2- 172.21.15.96:0/564361272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c104280 0x7f7a5c110650 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009ca0 tx=0x7f7a5402a8c0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 -- 172.21.15.96:0/564361272 shutdown_connections 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 --2- 172.21.15.96:0/564361272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c104280 0x7f7a5c110650 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 --2- 172.21.15.96:0/564361272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a5c109fd0 0x7f7a5c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 --2- 172.21.15.96:0/564361272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a5c109b30 0x7f7a5c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 -- 172.21.15.96:0/564361272 >> 172.21.15.96:0/564361272 conn(0x7f7a5c0ff210 msgr2=0x7f7a5c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 -- 172.21.15.96:0/564361272 shutdown_connections 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 -- 172.21.15.96:0/564361272 wait complete. 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 Processor -- start 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.970+0000 7f7a62e42640 1 -- start start 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a5c104280 0x7f7a5c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c109b30 0x7f7a5c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a5c109fd0 0x7f7a5c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c113d40 con 0x7f7a5c109b30 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c113bc0 con 0x7f7a5c109fd0 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c113ec0 con 0x7f7a5c104280 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c109b30 0x7f7a5c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c109b30 0x7f7a5c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59800/0 (socket says 172.21.15.96:59800) 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 -- 172.21.15.96:0/2642106808 learned_addr learned my addr 172.21.15.96:0/2642106808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a613df640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a5c109fd0 0x7f7a5c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a60bde640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a5c104280 0x7f7a5c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 -- 172.21.15.96:0/2642106808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a5c104280 msgr2=0x7f7a5c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a5c104280 0x7f7a5c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 -- 172.21.15.96:0/2642106808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a5c109fd0 msgr2=0x7f7a5c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:19.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a5c109fd0 0x7f7a5c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 -- 172.21.15.96:0/2642106808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c19f540 con 0x7f7a5c109b30 2024-04-25T17:20:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a5bfff640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c109b30 0x7f7a5c19ac40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c970 tx=0x7f7a4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a59ffb640 1 -- 172.21.15.96:0/2642106808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48004220 con 0x7f7a5c109b30 2024-04-25T17:20:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a59ffb640 1 -- 172.21.15.96:0/2642106808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a480043c0 con 0x7f7a5c109b30 2024-04-25T17:20:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c19f830 con 0x7f7a5c109b30 2024-04-25T17:20:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a59ffb640 1 -- 172.21.15.96:0/2642106808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800f450 con 0x7f7a5c109b30 2024-04-25T17:20:19.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.971+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c1a71b0 con 0x7f7a5c109b30 2024-04-25T17:20:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.972+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c06a190 con 0x7f7a5c109b30 2024-04-25T17:20:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.974+0000 7f7a59ffb640 1 -- 172.21.15.96:0/2642106808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7a480079b0 con 0x7f7a5c109b30 2024-04-25T17:20:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.974+0000 7f7a59ffb640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a4c0afc60 0x7f7a4c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.975+0000 7f7a60bde640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a4c0afc60 0x7f7a4c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.975+0000 7f7a59ffb640 1 -- 172.21.15.96:0/2642106808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f7a480df100 con 0x7f7a5c109b30 2024-04-25T17:20:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.975+0000 7f7a60bde640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a4c0afc60 0x7f7a4c0b2170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005fd0 tx=0x7f7a50005850 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:19.978+0000 7f7a59ffb640 1 -- 172.21.15.96:0/2642106808 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a48014030 con 0x7f7a5c109b30 2024-04-25T17:20:19.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:19 smithi097 ceph-mon[32602]: from='client.15759 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:20:19.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:19 smithi097 ceph-mon[32602]: from='client.15765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:20:19.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:19 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1066742963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:19.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:19 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/237430898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:20:19.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:19 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3051271924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:20:20.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:19 smithi171 ceph-mon[33189]: from='client.15759 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:20:20.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:19 smithi171 ceph-mon[33189]: from='client.15765 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T17:20:20.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:19 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1066742963' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:20.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:19 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/237430898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:20:20.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:19 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3051271924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:20:20.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.207+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7a5c02db50 con 0x7f7a5c109b30 2024-04-25T17:20:20.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.216+0000 7f7a59ffb640 1 -- 172.21.15.96:0/2642106808 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+803658 (secure 0 0 0) 0x7f7a480a7a30 con 0x7f7a5c109b30 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 26, 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "active_gid": 15261, 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "active_name": "x", 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addrs": { 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T17:20:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1594342167 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1594342167 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addr": "172.21.15.96:6801/1594342167", 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "active_change": "2024-04-25T17:10:14.788808+0000", 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T17:20:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "available": true, 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "standbys": [ 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24874, 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "z", 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T17:20:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T17:20:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T17:20:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T17:20:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T17:20:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T17:20:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T17:20:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T17:20:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T17:20:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T17:20:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T17:20:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T17:20:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T17:20:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T17:20:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T17:20:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T17:20:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T17:20:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T17:20:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T17:20:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T17:20:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T17:20:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.340 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T17:20:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T17:20:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T17:20:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T17:20:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T17:20:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T17:20:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T17:20:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T17:20:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T17:20:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T17:20:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T17:20:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T17:20:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T17:20:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T17:20:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T17:20:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T17:20:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T17:20:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T17:20:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T17:20:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.580 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T17:20:20.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T17:20:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T17:20:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T17:20:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T17:20:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T17:20:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T17:20:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24893, 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "y", 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-04-25T17:20:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T17:20:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T17:20:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T17:20:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T17:20:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T17:20:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T17:20:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T17:20:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T17:20:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T17:20:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T17:20:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T17:20:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T17:20:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T17:20:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T17:20:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T17:20:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T17:20:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T17:20:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T17:20:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T17:20:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T17:20:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T17:20:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T17:20:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T17:20:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T17:20:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T17:20:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T17:20:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T17:20:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T17:20:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T17:20:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T17:20:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.832 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T17:20:20.848 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.850 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.851 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.858 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.860 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T17:20:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T17:20:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T17:20:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T17:20:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T17:20:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.923 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T17:20:20.927 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.928 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.929 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.930 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.931 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.932 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T17:20:20.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.934 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.936 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.938 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.939 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.940 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.942 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.943 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.949 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.950 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.953 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.954 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T17:20:20.955 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.956 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.966 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T17:20:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T17:20:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.993 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:20.994 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:20.995 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:20.996 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.998 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:20.999 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.001 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.002 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.004 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.006 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.010 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.011 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.012 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T17:20:21.013 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.014 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.016 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.017 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.018 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.019 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T17:20:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T17:20:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T17:20:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T17:20:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T17:20:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T17:20:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T17:20:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T17:20:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T17:20:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "modules": [ 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard", 2024-04-25T17:20:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "iostat", 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "nfs", 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "restful", 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_schedule" 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T17:20:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T17:20:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T17:20:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T17:20:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T17:20:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T17:20:21.139 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.140 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.141 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.142 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.143 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.145 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T17:20:21.146 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.149 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T17:20:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.154 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:21.156 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T17:20:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T17:20:21.159 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T17:20:21.160 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.161 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.162 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T17:20:21.163 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T17:20:21.164 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T17:20:21.165 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T17:20:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T17:20:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T17:20:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T17:20:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T17:20:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T17:20:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.238 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T17:20:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T17:20:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T17:20:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T17:20:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:21.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.288 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.290 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.296 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T17:20:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T17:20:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T17:20:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T17:20:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T17:20:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T17:20:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T17:20:21.343 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T17:20:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T17:20:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T17:20:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T17:20:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T17:20:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T17:20:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T17:20:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T17:20:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.458 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T17:20:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T17:20:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T17:20:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.518 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T17:20:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T17:20:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "sqlite3_killpoint": { 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sqlite3_killpoint", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "dev", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T17:20:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T17:20:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "services": { 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard": "https://172.21.15.96:8443/" 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "always_on_modules": { 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "octopus": [ 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T17:20:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "pacific": [ 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T17:20:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "quincy": [ 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "reef": [ 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T17:20:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "squid": [ 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T17:20:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "last_failure_osd_epoch": 79, 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "active_clients": [ 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 4020135998 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "libcephsqlite", 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 4204787552 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1417021035 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1852753892 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T17:20:21.554 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 977218903 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1139473777 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T17:20:21.555 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T17:20:21.556 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:20 smithi171 ceph-mon[33189]: pgmap v314: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 245 B/s wr, 2 op/s 2024-04-25T17:20:21.556 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:20 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/448176426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:20:21.557 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:20 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2642106808' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T17:20:21.557 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:20 smithi097 ceph-mon[32602]: pgmap v314: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 245 B/s wr, 2 op/s 2024-04-25T17:20:21.557 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:20 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/448176426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:20:21.557 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:20 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2642106808' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a4c0afc60 msgr2=0x7f7a4c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a4c0afc60 0x7f7a4c0b2170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005fd0 tx=0x7f7a50005850 comp rx=0 tx=0).stop 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c109b30 msgr2=0x7f7a5c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c109b30 0x7f7a5c19ac40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c970 tx=0x7f7a4800ce30 comp rx=0 tx=0).stop 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 shutdown_connections 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7a4c0afc60 0x7f7a4c0b2170 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a5c109fd0 0x7f7a5c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a5c109b30 0x7f7a5c19ac40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 --2- 172.21.15.96:0/2642106808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a5c104280 0x7f7a5c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:21.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 >> 172.21.15.96:0/2642106808 conn(0x7f7a5c0ff210 msgr2=0x7f7a5c1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:21.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.221+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 shutdown_connections 2024-04-25T17:20:21.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:20.222+0000 7f7a62e42640 1 -- 172.21.15.96:0/2642106808 wait complete. 2024-04-25T17:20:21.564 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T17:20:21.564 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T17:20:21.565 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:20 smithi096 ceph-mon[29131]: pgmap v314: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 245 B/s wr, 2 op/s 2024-04-25T17:20:21.565 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:20 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/448176426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:20:21.565 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:20 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2642106808' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T17:20:21.814 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:20:22.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:22 smithi096 ceph-mon[29131]: pgmap v315: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:22.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:22 smithi097 ceph-mon[32602]: pgmap v315: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:23.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:22 smithi171 ceph-mon[33189]: pgmap v315: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001084209, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000141032, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 6, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 154, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3264, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 184176, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.012052828, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.007851146, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.002246720, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 152, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.015057175, 2024-04-25T17:20:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000099060 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 184176 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 211, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 200, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1684288, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 122784, 2024-04-25T17:20:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.048264163, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.013364186, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.043603819, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004139421, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 199, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.010076919, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000050637 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1684288, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 122784 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148955, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 30, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T17:20:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 213, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1654544, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 213, 2024-04-25T17:20:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 213, 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1654544, 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:23.677 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T17:20:24.441 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T17:20:24.441 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T17:20:24.441 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T17:20:24.441 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-25T17:20:24.442 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T17:20:24.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T17:20:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:20:24.928 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:24 smithi096 ceph-mon[29131]: pgmap v316: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:24.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:24 smithi097 ceph-mon[32602]: pgmap v316: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:25.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:24 smithi171 ceph-mon[33189]: pgmap v316: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:26.446 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001241444, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000276763, 2024-04-25T17:20:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 217, 2024-04-25T17:20:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 206, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1685920, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 126624, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.048382035, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.013804680, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.044044287, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004136281, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 205, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009610756, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000046881 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1685920, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 126624 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 6, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 159, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3264, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 185056, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.013623237, 2024-04-25T17:20:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.009163184, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.002290241, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 157, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.016628553, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000105914 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 185056 2024-04-25T17:20:26.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:26.452 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:26.453 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 149017, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 31, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:26.454 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T17:20:26.455 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 219, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1655324, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 219, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T17:20:26.456 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 219, 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1655324, 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:26.457 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T17:20:26.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:26 smithi096 ceph-mon[29131]: pgmap v317: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:26.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:26 smithi097 ceph-mon[32602]: pgmap v317: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:27.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:26 smithi171 ceph-mon[33189]: pgmap v317: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T17:20:27.162 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T17:20:27.162 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T17:20:27.162 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T17:20:27.162 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T17:20:27.162 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T17:20:27.162 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T17:20:27.402 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:20:28.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:28 smithi096 ceph-mon[29131]: pgmap v318: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T17:20:28.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:28 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:20:28.965 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:28 smithi171 ceph-mon[33189]: pgmap v318: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T17:20:28.965 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:28 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:20:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:28 smithi097 ceph-mon[32602]: pgmap v318: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T17:20:28.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:28 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 213, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 202, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1684832, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 124064, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.048732181, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.012941447, 2024-04-25T17:20:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.043362271, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004191595, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 201, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009733557, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000048425 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1684832, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 124064 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 154, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 184080, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.012904867, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.008549245, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.002105069, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 151, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.016310467, 2024-04-25T17:20:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000108016 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 184080 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2944, 2024-04-25T17:20:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.000751337, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000033860, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000707258, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000925973, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000308657 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148994, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 31, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T17:20:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T17:20:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 215, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1654804, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 215, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 215, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1654804, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:29.135 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T17:20:29.851 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T17:20:29.851 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T17:20:29.851 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T17:20:29.851 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-25T17:20:29.851 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T17:20:29.852 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T17:20:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T17:20:30.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: pgmap v319: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:30 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: pgmap v319: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:30.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:30 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: pgmap v319: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T17:20:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.032 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:30 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:31.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:31 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T17:20:31.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:31 smithi096 ceph-mon[29131]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T17:20:31.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:31 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T17:20:31.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:31 smithi096 ceph-mon[29131]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:20:31.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:31 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T17:20:31.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:31 smithi097 ceph-mon[32602]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T17:20:31.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:31 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T17:20:31.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:31 smithi097 ceph-mon[32602]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:20:32.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:31 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-25T17:20:32.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:31 smithi171 ceph-mon[33189]: Unable to set osd_memory_target on smithi171 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-25T17:20:32.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:31 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T17:20:32.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:31 smithi171 ceph-mon[33189]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:20:32.407 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T17:20:32.408 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:32.408 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:32.408 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:32.408 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:32.408 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:32.408 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:32.408 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:32.409 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001088806, 2024-04-25T17:20:32.409 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:32.409 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000193909, 2024-04-25T17:20:32.409 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:32.409 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:32.409 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:32.410 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:32.410 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:32.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.410 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:32.410 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:32.410 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:32.411 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:32.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.411 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:32.411 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:32.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.411 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:32.412 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 6, 2024-04-25T17:20:32.412 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 161, 2024-04-25T17:20:32.412 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3264, 2024-04-25T17:20:32.412 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 185408, 2024-04-25T17:20:32.412 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:32.412 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:32.413 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.013969451, 2024-04-25T17:20:32.413 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.009422360, 2024-04-25T17:20:32.413 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.002439169, 2024-04-25T17:20:32.413 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:32.413 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:32.413 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 159, 2024-04-25T17:20:32.414 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.013183106, 2024-04-25T17:20:32.414 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000082912 2024-04-25T17:20:32.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.414 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:32.414 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:32.414 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:32.415 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:32.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.415 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T17:20:32.415 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 185408 2024-04-25T17:20:32.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.415 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:32.416 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 219, 2024-04-25T17:20:32.416 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 208, 2024-04-25T17:20:32.416 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1686464, 2024-04-25T17:20:32.416 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 127904, 2024-04-25T17:20:32.416 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:32.416 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:32.417 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.046600577, 2024-04-25T17:20:32.417 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.013080436, 2024-04-25T17:20:32.417 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.043861335, 2024-04-25T17:20:32.417 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004282095, 2024-04-25T17:20:32.417 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:32.417 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 207, 2024-04-25T17:20:32.418 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009762494, 2024-04-25T17:20:32.418 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000047161 2024-04-25T17:20:32.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.418 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:32.419 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:32.419 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:32.419 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:32.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.419 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1686464, 2024-04-25T17:20:32.419 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 127904 2024-04-25T17:20:32.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.420 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T17:20:32.420 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T17:20:32.420 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T17:20:32.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.420 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T17:20:32.420 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:32.421 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:32.421 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:32.421 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:32.421 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:32.421 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:32.421 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:32.422 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:32.422 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:32.422 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:32.422 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:32.422 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:32.423 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:32.423 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:32.423 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:32.423 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:32.423 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:32.423 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:32.423 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:32.424 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:32.424 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:32.424 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:32.424 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:32.424 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:32.424 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:32.424 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:32.425 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:32.425 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:32.425 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:32.425 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:32.425 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:32.425 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T17:20:32.425 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:32.426 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:32.426 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:32.426 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:32.426 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148977, 2024-04-25T17:20:32.426 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 30, 2024-04-25T17:20:32.426 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:32.426 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:32.427 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T17:20:32.427 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T17:20:32.427 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:32.427 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:32.427 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T17:20:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T17:20:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T17:20:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T17:20:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T17:20:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T17:20:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T17:20:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T17:20:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T17:20:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 221, 2024-04-25T17:20:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1655584, 2024-04-25T17:20:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 221, 2024-04-25T17:20:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T17:20:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T17:20:32.432 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 221, 2024-04-25T17:20:32.432 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1655584, 2024-04-25T17:20:32.432 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T17:20:32.432 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T17:20:32.432 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T17:20:32.432 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T17:20:32.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:32.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T17:20:32.433 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T17:20:32.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:32 smithi096 ceph-mon[29131]: pgmap v320: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T17:20:32.984 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:32 smithi097 ceph-mon[32602]: pgmap v320: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T17:20:33.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:32 smithi171 ceph-mon[33189]: pgmap v320: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T17:20:33.093 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T17:20:33.093 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T17:20:33.093 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T17:20:33.093 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T17:20:33.093 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T17:20:33.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T17:20:33.342 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:20:34.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: pgmap v321: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:34.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:20:34.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:20:34.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:34 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: pgmap v321: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:34.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.477 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.478 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:20:34.478 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:20:34.478 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:34 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: pgmap v321: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T17:20:34.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T17:20:34.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T17:20:34.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:34 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' 2024-04-25T17:20:35.203 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T17:20:35.203 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:35.203 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 28565, 2024-04-25T17:20:35.203 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 28021, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 7497537, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 75914090, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 16, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 1.916169911, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.677393239, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.446394434, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.677645155, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 28019, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 6.908242922, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000246555 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.204 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 840, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000168553, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000200 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 302592, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 87536 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 32909, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 32763, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 9114263, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 94385308, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 48, 2024-04-25T17:20:35.205 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 8, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 2.121069911, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.744011529, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.495217509, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.747428829, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 32760, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 4.794535926, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000146353 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 246, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000044691, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000181 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.206 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 86448, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 135920 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 51016, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 101955, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 18773800, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 50183740, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 12, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 4.156902353, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 1.857759974, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 6.259134561, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.373950529, 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:35.207 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 101955, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 5.009151862, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000049131 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 103801, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.016967929, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000163 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 15263312, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 42661264 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T17:20:35.208 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 88262, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 31.522118923, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000357142 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4693, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.679124903, 2024-04-25T17:20:35.209 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000570876 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 20663, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 20663, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 20663, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 67.067212497, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003245763 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T17:20:35.210 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 969, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 22, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 9950, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 3050, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 6900, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 5592, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 3, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 4, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 21807, 2024-04-25T17:20:35.211 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 21279, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 4, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T17:20:35.212 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 8562, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 19364, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 7863, 2024-04-25T17:20:35.213 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 15597, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 12379, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 443, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 14827 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 62, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T17:20:35.214 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 5172, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 5108, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 3, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T17:20:35.215 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 22, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 51916, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 43816, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 43816, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 8100, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 6790, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 87, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 76, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 76, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 11, 2024-04-25T17:20:35.216 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 1, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 11, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 131119580, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 148948429, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 24985, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 58.397488005, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002337301 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 4680, 2024-04-25T17:20:35.217 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 9787, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 5107, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 233, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 261, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 28, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 9950, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 14966, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 5016, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 4, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 5189, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 5185, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 189608, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 231824 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.218 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 38980, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 20663, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 33, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5100, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 19.629582156, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003848937 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 105, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.021385821, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000203674 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.219 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.005546877, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001848959 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 251, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.133695913, 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000532653 2024-04-25T17:20:35.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000268064, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000134032 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001700628, 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000170062 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 73, 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.068800131, 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000942467 2024-04-25T17:20:35.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.051343264, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005134326 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 415, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.101701983, 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000245065 2024-04-25T17:20:35.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 470, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.031504804, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002194691 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 92, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.211285628, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002296582 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.225 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 67, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.617968506, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.039074156 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2024-04-25T17:20:35.226 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.015596209, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003899052 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8994, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 33.727193444, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003749965 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T17:20:35.227 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8, 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.262933480, 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.032866685 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.228 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 7, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.020306465, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002900923 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 6, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003997417, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000666236 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.229 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5046, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 9.162401707, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001815775 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.230 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 5, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 3, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 2, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 361, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 180, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 599, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:35.231 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:35.232 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:35.233 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 976021, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 4897, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 1584, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 18, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:35.234 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8128, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 169, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 18341903, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 190870, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T17:20:35.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 73977, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 157264059, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 73977, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 73977, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1061.090641952, 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.014343520 2024-04-25T17:20:35.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 73977, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 86214 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 73977, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 57, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 73920, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 714, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 5186, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 57, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 25042, 2024-04-25T17:20:35.237 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 121, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 1, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 6, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 5728, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 10373, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T17:20:35.238 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 36717, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T17:20:35.239 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 88, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 13, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 2105, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T17:20:35.240 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 164 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 56, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T17:20:35.241 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 32, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 5137, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 5108, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 112486, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 23465226, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 112486, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:35.242 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 112486, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 23465226, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:35.243 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 73977, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 157889711, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 73977, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 157889711, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:35.244 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 73977, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 73977, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 73977, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 73977, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.245 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 5108, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 625652, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 5108, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 57, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 625652, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.246 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55d2e78ba8f0": { 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 51916, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 144754125, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 51916, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 24985, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 144754125, 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:35.247 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:35.248 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:35.248 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:35.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.248 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:35.248 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T17:20:35.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:35 smithi096 ceph-mon[29131]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T17:20:35.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:35 smithi096 ceph-mon[29131]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:20:35.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:35 smithi097 ceph-mon[32602]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T17:20:35.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:35 smithi097 ceph-mon[32602]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:20:35.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:35 smithi171 ceph-mon[33189]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T17:20:35.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:35 smithi171 ceph-mon[33189]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T17:20:36.075 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T17:20:36.075 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T17:20:36.075 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T17:20:36.075 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-25T17:20:36.075 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T17:20:36.076 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T17:20:36.315 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:20:36.437 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:36 smithi097 ceph-mon[32602]: pgmap v322: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:36.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:36 smithi096 ceph-mon[29131]: pgmap v322: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:36.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:36 smithi171 ceph-mon[33189]: pgmap v322: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:38.170 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T17:20:38.170 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.001092795, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000285126, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:38.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 219, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 208, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1686464, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 127904, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:38.172 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.044570910, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.012483781, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.038055871, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.005091671, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 207, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.010594846, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000051182 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1686464, 2024-04-25T17:20:38.173 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 127904 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 6, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 161, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3264, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 185408, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.010753299, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006768955, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.002058195, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 159, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.017319546, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000108927 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 185408 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-04-25T17:20:38.175 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:38.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:38.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148994, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 31, 2024-04-25T17:20:38.178 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T17:20:38.179 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 221, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1655584, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 221, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:38.180 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 221, 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1655584, 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:38.181 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T17:20:38.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:38 smithi096 ceph-mon[29131]: pgmap v323: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:38.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:38 smithi097 ceph-mon[32602]: pgmap v323: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:38.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:38 smithi171 ceph-mon[33189]: pgmap v323: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:38.827 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T17:20:38.827 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T17:20:38.827 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T17:20:38.827 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-25T17:20:38.827 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T17:20:38.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T17:20:39.064 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:20:40.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:40 smithi096 ceph-mon[29131]: pgmap v324: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:40.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:40 smithi097 ceph-mon[32602]: pgmap v324: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:40.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:40 smithi171 ceph-mon[33189]: pgmap v324: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:40.843 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T17:20:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 710, 2024-04-25T17:20:40.843 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 712, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 50554522, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 225790, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.085270771, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.024892886, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.034980418, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.018677913, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 710, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.029891728, 2024-04-25T17:20:40.844 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000042101 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 875, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 971, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 47133719, 2024-04-25T17:20:40.845 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 537405, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.091141645, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.021394422, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.041679502, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.022564704, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 970, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.034914942, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000035994 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:40.846 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 260688 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 261, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 260, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 25401628, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 118864, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2024-04-25T17:20:40.847 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.041523475, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.010050548, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.024233002, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.005769842, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 259, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.011696465, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000045160 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.848 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 203376, 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 81648 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2805, 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.119531931, 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000042613 2024-04-25T17:20:40.849 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-25T17:20:40.850 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T17:20:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 14452, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 4433, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 10019, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 8120, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 2, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T17:20:40.852 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2024-04-25T17:20:40.853 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T17:20:40.854 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T17:20:40.855 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T17:20:40.856 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 41932, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T17:20:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 11, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 13, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 131119580, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 148948429, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 148948317, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 41932 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 4654, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 9667, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 5013, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 228, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 230, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 2, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 14452, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 14458, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 6, 2024-04-25T17:20:40.859 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 149676, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215440 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.860 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.861 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T17:20:40.863 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T17:20:40.865 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T17:20:40.867 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T17:20:40.868 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.869 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.870 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T17:20:40.871 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 1732, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:40.872 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:40.873 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:40.874 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 412822, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 4806, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 8128, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 169, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T17:20:40.875 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 21167510, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 173999, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 1697, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 26, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 4, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 1697, 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T17:20:40.876 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1697, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.285977942, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000757794 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1697, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1698 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 1697, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 1697, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T17:20:40.877 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1012, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 682, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 2, 2024-04-25T17:20:40.878 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T17:20:40.879 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 88, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 8, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 11, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 11, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T17:20:40.880 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T17:20:40.881 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1842, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 122913765, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1842, 2024-04-25T17:20:40.882 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1842, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 122913765, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:40.883 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 1697, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 122409380, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1697, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 122409380, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:40.884 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 1697, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 1697, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1697, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1697, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:40.886 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55a2803feea0": { 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:40.887 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:40.888 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T17:20:41.678 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T17:20:41.678 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T17:20:41.678 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T17:20:41.678 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T17:20:41.678 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T17:20:41.679 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T17:20:41.914 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T17:20:42.388 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:42 smithi097 ceph-mon[32602]: pgmap v325: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:42.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:42 smithi096 ceph-mon[29131]: pgmap v325: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:42.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:42 smithi171 ceph-mon[33189]: pgmap v325: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:43.758 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T17:20:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:43.758 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.001083550, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000302064, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.759 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 227, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 214, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1690560, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 130432, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.049128621, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.013212163, 2024-04-25T17:20:43.760 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.043459073, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.005138728, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 214, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.010799897, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000050466 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1690560, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 130432 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2, 2024-04-25T17:20:43.761 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 162, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 320, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 185488, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.010081221, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006495114, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001678718, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 159, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.017745587, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000111607 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:43.762 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 185488 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:43.763 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:43.764 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 149034, 2024-04-25T17:20:43.765 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 32, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T17:20:43.766 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 225, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1656168, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 225, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T17:20:43.767 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 225, 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1656168, 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T17:20:43.768 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T17:20:44.402 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:44 smithi097 ceph-mon[32602]: pgmap v326: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:44.427 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T17:20:44.428 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T17:20:44.428 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T17:20:44.428 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-25T17:20:44.428 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T17:20:44.428 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T17:20:44.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:44 smithi096 ceph-mon[29131]: pgmap v326: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:44.454 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:44 smithi171 ceph-mon[33189]: pgmap v326: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:44.679 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:20:46.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:46 smithi171 ceph-mon[33189]: pgmap v327: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:46.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:46 smithi096 ceph-mon[29131]: pgmap v327: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:46.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:46 smithi097 ceph-mon[32602]: pgmap v327: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:46.546 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T17:20:46.546 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:46.546 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 220, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 209, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1686800, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 128064, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.050077275, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.013029072, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.043112174, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.005626104, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 208, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.011103279, 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000053381 2024-04-25T17:20:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1686800, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 128064 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 6, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 162, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3264, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 185584, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:46.548 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.014409571, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007762148, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.004508192, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 160, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.016948347, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000105927 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.549 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 185584 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000370009, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000297694, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:46.550 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:46.551 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:46.552 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:46.553 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T17:20:46.554 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 149017, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 31, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-25T17:20:46.555 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T17:20:46.556 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 222, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1655778, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 222, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 222, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1655778, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:46.557 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T17:20:47.232 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-25T17:20:47.232 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-25T17:20:47.233 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-25T17:20:47.233 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.c 2024-04-25T17:20:47.233 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T17:20:47.233 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T17:20:47.530 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:20:48.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:48 smithi096 ceph-mon[29131]: pgmap v328: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:48.478 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:48 smithi171 ceph-mon[33189]: pgmap v328: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:48.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:48 smithi097 ceph-mon[32602]: pgmap v328: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:49.339 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 226, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 213, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1690288, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 129792, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.049911243, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.013690425, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.042697409, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.005358669, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 213, 2024-04-25T17:20:49.340 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.011948563, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000056096 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1690288, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 129792 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 160, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 320, 2024-04-25T17:20:49.341 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 185136, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.010580651, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.006917728, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001655083, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 157, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.014941576, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000095169 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:49.342 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 185136 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:49.343 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000357082, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000278036, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:49.344 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:49.345 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:49.346 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:49.347 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 149034, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 32, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:49.348 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 224, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1656038, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:49.349 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 224, 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 224, 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1656038, 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:49.350 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T17:20:50.212 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-25T17:20:50.212 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-25T17:20:50.212 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-25T17:20:50.212 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.f 2024-04-25T17:20:50.212 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T17:20:50.213 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T17:20:50.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:50 smithi096 ceph-mon[29131]: pgmap v329: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:50.458 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:20:50.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:50 smithi097 ceph-mon[32602]: pgmap v329: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:50.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:50 smithi171 ceph-mon[33189]: pgmap v329: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:52.154 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:52.155 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.001204064, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000264175, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:52.156 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 220, 2024-04-25T17:20:52.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 209, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1686800, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 128064, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.045909228, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.013304189, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.038562474, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.005096120, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 208, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.010513595, 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000050546 2024-04-25T17:20:52.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1686800, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 128064 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 6, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 162, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3264, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 185584, 2024-04-25T17:20:52.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.011415682, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007328664, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.002183516, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 160, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.015387778, 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000096173 2024-04-25T17:20:52.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 185584 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:52.161 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:52.162 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:52.163 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148994, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 31, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:52.164 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:52.165 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 222, 2024-04-25T17:20:52.166 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1655778, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 222, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 222, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1655778, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:52.167 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:52.168 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T17:20:52.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:52 smithi096 ceph-mon[29131]: pgmap v330: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:52.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:52 smithi097 ceph-mon[32602]: pgmap v330: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:52.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:52 smithi171 ceph-mon[33189]: pgmap v330: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:52.818 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2024-04-25T17:20:52.818 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2024-04-25T17:20:52.818 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2024-04-25T17:20:52.819 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.i 2024-04-25T17:20:52.819 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T17:20:52.819 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid 1be09d02-0325-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T17:20:53.060 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T17:20:54.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:54 smithi171 ceph-mon[33189]: pgmap v331: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:54.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:54 smithi096 ceph-mon[29131]: pgmap v331: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:54.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:54 smithi097 ceph-mon[32602]: pgmap v331: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:54.839 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 227, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 216, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1688704, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 132544, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.050506962, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.014206024, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.042256264, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.005655036, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 215, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.011534186, 2024-04-25T17:20:54.840 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000053647 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1688704, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 132544 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 6, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 166, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3264, 2024-04-25T17:20:54.841 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 186288, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.011496992, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.007413875, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001943657, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 164, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.019835910, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000120950 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:54.842 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 186288 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000346924, 2024-04-25T17:20:54.843 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000287421, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.844 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-04-25T17:20:54.845 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T17:20:54.846 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T17:20:54.846 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T17:20:54.846 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T17:20:54.846 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-04-25T17:20:54.847 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 149007, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 30, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-04-25T17:20:54.848 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-04-25T17:20:54.849 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 229, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1656688, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 229, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 229, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1656688, 2024-04-25T17:20:54.850 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-04-25T17:20:54.851 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-04-25T17:20:54.851 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-04-25T17:20:54.851 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-04-25T17:20:54.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:54.851 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-04-25T17:20:54.851 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-25T17:20:55.501 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T17:20:55.501 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T17:20:55.501 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T17:20:55.501 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-25T17:20:55.502 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T17:20:55.512 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T17:20:55.512 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:20:55.512 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T17:20:55.532 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T17:20:55.533 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T17:20:55.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:20:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 -- 172.21.15.96:0/1681870158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 msgr2=0x7f9fc4102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/1681870158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc4102f00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009900 tx=0x7f9fb402a410 comp rx=0 tx=0).stop 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 -- 172.21.15.96:0/1681870158 shutdown_connections 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/1681870158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc410d380 0x7f9fc410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/1681870158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc4102f00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/1681870158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc4108a20 0x7f9fc41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 -- 172.21.15.96:0/1681870158 >> 172.21.15.96:0/1681870158 conn(0x7f9fc4078110 msgr2=0x7f9fc4078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 -- 172.21.15.96:0/1681870158 shutdown_connections 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.751+0000 7f9fc94f9640 1 -- 172.21.15.96:0/1681870158 wait complete. 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 Processor -- start 2024-04-25T17:20:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 -- start start 2024-04-25T17:20:55.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:55.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc4108a20 0x7f9fc4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc410d380 0x7f9fc419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4112be0 con 0x7f9fc4102af0 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4112a60 con 0x7f9fc4108a20 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc94f9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc4112d60 con 0x7f9fc410d380 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc41995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40738/0 (socket says 172.21.15.96:40738) 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.752+0000 7f9fc2ffd640 1 -- 172.21.15.96:0/770915436 learned_addr learned my addr 172.21.15.96:0/770915436 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc37fe640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc410d380 0x7f9fc419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc27fc640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc4108a20 0x7f9fc4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc2ffd640 1 -- 172.21.15.96:0/770915436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc410d380 msgr2=0x7f9fc419de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc2ffd640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc410d380 0x7f9fc419de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc2ffd640 1 -- 172.21.15.96:0/770915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc4108a20 msgr2=0x7f9fc4199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:55.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc2ffd640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc4108a20 0x7f9fc4199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc2ffd640 1 -- 172.21.15.96:0/770915436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc419e440 con 0x7f9fc4102af0 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc2ffd640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc41995e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800c970 tx=0x7f9fb800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fabfff640 1 -- 172.21.15.96:0/770915436 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8007bb0 con 0x7f9fc4102af0 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc419e730 con 0x7f9fc4102af0 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fabfff640 1 -- 172.21.15.96:0/770915436 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fb8007d50 con 0x7f9fc4102af0 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fabfff640 1 -- 172.21.15.96:0/770915436 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800f450 con 0x7f9fc4102af0 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.753+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc41a60b0 con 0x7f9fc4102af0 2024-04-25T17:20:55.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.754+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc4103cd0 con 0x7f9fc4102af0 2024-04-25T17:20:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.756+0000 7f9fabfff640 1 -- 172.21.15.96:0/770915436 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f9fb8004ea0 con 0x7f9fc4102af0 2024-04-25T17:20:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.757+0000 7f9fabfff640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f9fac0afc60 0x7f9fac0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.757+0000 7f9fc27fc640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f9fac0afc60 0x7f9fac0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.757+0000 7f9fabfff640 1 -- 172.21.15.96:0/770915436 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f9fb80df1b0 con 0x7f9fc4102af0 2024-04-25T17:20:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.757+0000 7f9fc27fc640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f9fac0afc60 0x7f9fac0b2170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009230 tx=0x7f9fb4031400 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.761+0000 7f9fabfff640 1 -- 172.21.15.96:0/770915436 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fb8014030 con 0x7f9fc4102af0 2024-04-25T17:20:55.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.940+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fc4106ee0 con 0x7f9fc4102af0 2024-04-25T17:20:55.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.942+0000 7f9fabfff640 1 -- 172.21.15.96:0/770915436 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23060 (secure 0 0 0) 0x7f9fb80a7a40 con 0x7f9fc4102af0 2024-04-25T17:20:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:20:55.943 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:20:18.541083+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/559998201":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/1944193211":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/3419689135":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6801/3530143863":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6800/3530143863":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/328848234":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:20:55.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f9fac0afc60 msgr2=0x7f9fac0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f9fac0afc60 0x7f9fac0b2170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009230 tx=0x7f9fb4031400 comp rx=0 tx=0).stop 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 msgr2=0x7f9fc41995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc41995e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800c970 tx=0x7f9fb800ce30 comp rx=0 tx=0).stop 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 shutdown_connections 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc410d380 0x7f9fc419de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f9fac0afc60 0x7f9fac0b2170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fc4108a20 0x7f9fc4199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 --2- 172.21.15.96:0/770915436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9fc4102af0 0x7f9fc41995e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 >> 172.21.15.96:0/770915436 conn(0x7f9fc4078110 msgr2=0x7f9fc41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 shutdown_connections 2024-04-25T17:20:55.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:55.944+0000 7f9fc94f9640 1 -- 172.21.15.96:0/770915436 wait complete. 2024-04-25T17:20:55.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T17:03:27.048575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T17:08:09.266201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T17:08:10.452964+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T17:20:55.957 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T17:20:56.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 -- 172.21.15.96:0/155106221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010e570 msgr2=0x7f84a0110990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 --2- 172.21.15.96:0/155106221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010e570 0x7f84a0110990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8498009ca0 tx=0x7f849802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 -- 172.21.15.96:0/155106221 shutdown_connections 2024-04-25T17:20:56.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 --2- 172.21.15.96:0/155106221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a010e570 0x7f84a0110990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 --2- 172.21.15.96:0/155106221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a0102070 0x7f84a0102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 --2- 172.21.15.96:0/155106221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a0109bc0 0x7f84a0101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 -- 172.21.15.96:0/155106221 >> 172.21.15.96:0/155106221 conn(0x7f84a0077920 msgr2=0x7f84a0077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:56.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 -- 172.21.15.96:0/155106221 shutdown_connections 2024-04-25T17:20:56.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.066+0000 7f84a5092640 1 -- 172.21.15.96:0/155106221 wait complete. 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.067+0000 7f84a5092640 1 Processor -- start 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.067+0000 7f84a5092640 1 -- start start 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f84a5092640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a0102070 0x7f84a019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f84a5092640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0109bc0 0x7f84a019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f84a5092640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a010e570 0x7f84a019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f84a5092640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0113d30 con 0x7f84a0109bc0 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f84a5092640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0113bb0 con 0x7f84a0102070 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f84a5092640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0113eb0 con 0x7f84a010e570 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0109bc0 0x7f84a019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0109bc0 0x7f84a019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40764/0 (socket says 172.21.15.96:40764) 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 -- 172.21.15.96:0/724891079 learned_addr learned my addr 172.21.15.96:0/724891079 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:56.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849ed9d640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a0102070 0x7f84a019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849f59e640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a010e570 0x7f84a019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 -- 172.21.15.96:0/724891079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a010e570 msgr2=0x7f84a019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a010e570 0x7f84a019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 -- 172.21.15.96:0/724891079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a0102070 msgr2=0x7f84a019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a0102070 0x7f84a019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 -- 172.21.15.96:0/724891079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a019f4b0 con 0x7f84a0109bc0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f849e59c640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0109bc0 0x7f84a019ac30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f849400c9a0 tx=0x7f849400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f8487fff640 1 -- 172.21.15.96:0/724891079 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494014070 con 0x7f84a0109bc0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.068+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a019f7a0 con 0x7f84a0109bc0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.069+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a71a0 con 0x7f84a0109bc0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.069+0000 7f8487fff640 1 -- 172.21.15.96:0/724891079 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8494004440 con 0x7f84a0109bc0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.069+0000 7f8487fff640 1 -- 172.21.15.96:0/724891079 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494003d50 con 0x7f84a0109bc0 2024-04-25T17:20:56.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.070+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848c005290 con 0x7f84a0109bc0 2024-04-25T17:20:56.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.071+0000 7f8487fff640 1 -- 172.21.15.96:0/724891079 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8494017070 con 0x7f84a0109bc0 2024-04-25T17:20:56.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.071+0000 7f8487fff640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84880afc60 0x7f84880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.071+0000 7f849ed9d640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84880afc60 0x7f84880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.072+0000 7f8487fff640 1 -- 172.21.15.96:0/724891079 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f84940de7a0 con 0x7f84a0109bc0 2024-04-25T17:20:56.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.072+0000 7f849ed9d640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84880afc60 0x7f84880b2170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8490005fd0 tx=0x7f849000b410 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.076+0000 7f8487fff640 1 -- 172.21.15.96:0/724891079 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f849401e070 con 0x7f84a0109bc0 2024-04-25T17:20:56.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.253+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f848c005850 con 0x7f84a0109bc0 2024-04-25T17:20:56.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.254+0000 7f8487fff640 1 -- 172.21.15.96:0/724891079 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f84940a7030 con 0x7f84a0109bc0 2024-04-25T17:20:56.255 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84880afc60 msgr2=0x7f84880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84880afc60 0x7f84880b2170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8490005fd0 tx=0x7f849000b410 comp rx=0 tx=0).stop 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0109bc0 msgr2=0x7f84a019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0109bc0 0x7f84a019ac30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f849400c9a0 tx=0x7f849400ce60 comp rx=0 tx=0).stop 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 shutdown_connections 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84a010e570 0x7f84a019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84a0109bc0 0x7f84a019ac30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84880afc60 0x7f84880b2170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 --2- 172.21.15.96:0/724891079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84a0102070 0x7f84a019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 >> 172.21.15.96:0/724891079 conn(0x7f84a0077920 msgr2=0x7f84a0105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:56.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 shutdown_connections 2024-04-25T17:20:56.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.256+0000 7f84a5092640 1 -- 172.21.15.96:0/724891079 wait complete. 2024-04-25T17:20:56.267 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:56 smithi096 ceph-mon[29131]: pgmap v332: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:56.268 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:56 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/770915436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:56.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T17:20:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 -- 172.21.15.96:0/2001936698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 msgr2=0x7f1c7c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 --2- 172.21.15.96:0/2001936698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c1031c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009900 tx=0x7f1c6c02a410 comp rx=0 tx=0).stop 2024-04-25T17:20:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 -- 172.21.15.96:0/2001936698 shutdown_connections 2024-04-25T17:20:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 --2- 172.21.15.96:0/2001936698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c103830 0x7f1c7c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 --2- 172.21.15.96:0/2001936698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c1031c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 --2- 172.21.15.96:0/2001936698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c7c108a10 0x7f1c7c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 -- 172.21.15.96:0/2001936698 >> 172.21.15.96:0/2001936698 conn(0x7f1c7c077880 msgr2=0x7f1c7c077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 -- 172.21.15.96:0/2001936698 shutdown_connections 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.378+0000 7f1c846ca640 1 -- 172.21.15.96:0/2001936698 wait complete. 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 Processor -- start 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 -- start start 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c7c103830 0x7f1c7c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c108a10 0x7f1c7c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c112c00 con 0x7f1c7c108eb0 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c112a80 con 0x7f1c7c108a10 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c846ca640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7c112d80 con 0x7f1c7c103830 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c82c67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c82c67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40782/0 (socket says 172.21.15.96:40782) 2024-04-25T17:20:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.379+0000 7f1c82c67640 1 -- 172.21.15.96:0/370911225 learned_addr learned my addr 172.21.15.96:0/370911225 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82466640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c7c103830 0x7f1c7c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c81c65640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c108a10 0x7f1c7c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82c67640 1 -- 172.21.15.96:0/370911225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c7c103830 msgr2=0x7f1c7c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82c67640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c7c103830 0x7f1c7c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82c67640 1 -- 172.21.15.96:0/370911225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c108a10 msgr2=0x7f1c7c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82c67640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c108a10 0x7f1c7c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82c67640 1 -- 172.21.15.96:0/370911225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7c19e440 con 0x7f1c7c108eb0 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82466640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c7c103830 0x7f1c7c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c82c67640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c19de80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400cd10 tx=0x7f1c7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c737fe640 1 -- 172.21.15.96:0/370911225 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400f450 con 0x7f1c7c108eb0 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c737fe640 1 -- 172.21.15.96:0/370911225 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7400faa0 con 0x7f1c7c108eb0 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c737fe640 1 -- 172.21.15.96:0/370911225 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74018b50 con 0x7f1c7c108eb0 2024-04-25T17:20:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7c19e730 con 0x7f1c7c108eb0 2024-04-25T17:20:56.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.380+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7c19ec00 con 0x7f1c7c108eb0 2024-04-25T17:20:56.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.381+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7c103ea0 con 0x7f1c7c108eb0 2024-04-25T17:20:56.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.382+0000 7f1c737fe640 1 -- 172.21.15.96:0/370911225 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f1c7400fc50 con 0x7f1c7c108eb0 2024-04-25T17:20:56.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.383+0000 7f1c737fe640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f1c680afc60 0x7f1c680b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.383+0000 7f1c82466640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f1c680afc60 0x7f1c680b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.383+0000 7f1c737fe640 1 -- 172.21.15.96:0/370911225 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f1c740df7d0 con 0x7f1c7c108eb0 2024-04-25T17:20:56.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.384+0000 7f1c82466640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f1c680afc60 0x7f1c680b2170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005fd0 tx=0x7f1c78005850 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:56.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.387+0000 7f1c737fe640 1 -- 172.21.15.96:0/370911225 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c74014030 con 0x7f1c7c108eb0 2024-04-25T17:20:56.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:56 smithi097 ceph-mon[32602]: pgmap v332: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:56.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:56 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/770915436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:56.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:56 smithi171 ceph-mon[33189]: pgmap v332: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T17:20:56.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:56 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/770915436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:56.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.562+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c7c1a6190 con 0x7f1c7c108eb0 2024-04-25T17:20:56.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.562+0000 7f1c737fe640 1 -- 172.21.15.96:0/370911225 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f1c740a8060 con 0x7f1c7c108eb0 2024-04-25T17:20:56.563 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T17:20:56.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f1c680afc60 msgr2=0x7f1c680b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f1c680afc60 0x7f1c680b2170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005fd0 tx=0x7f1c78005850 comp rx=0 tx=0).stop 2024-04-25T17:20:56.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 msgr2=0x7f1c7c19de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c19de80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400cd10 tx=0x7f1c7400a620 comp rx=0 tx=0).stop 2024-04-25T17:20:56.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 shutdown_connections 2024-04-25T17:20:56.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c7c108eb0 0x7f1c7c19de80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f1c680afc60 0x7f1c680b2170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c108a10 0x7f1c7c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 --2- 172.21.15.96:0/370911225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c7c103830 0x7f1c7c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 >> 172.21.15.96:0/370911225 conn(0x7f1c7c077880 msgr2=0x7f1c7c100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:56.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 shutdown_connections 2024-04-25T17:20:56.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.564+0000 7f1c846ca640 1 -- 172.21.15.96:0/370911225 wait complete. 2024-04-25T17:20:56.576 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T17:20:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.686+0000 7feca5f7d640 1 -- 172.21.15.96:0/91810980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109af0 msgr2=0x7feca0101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.686+0000 7feca5f7d640 1 --2- 172.21.15.96:0/91810980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109af0 0x7feca0101b70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fec90009900 tx=0x7fec9002a410 comp rx=0 tx=0).stop 2024-04-25T17:20:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 -- 172.21.15.96:0/91810980 shutdown_connections 2024-04-25T17:20:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 --2- 172.21.15.96:0/91810980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0102720 0x7feca0110570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 --2- 172.21.15.96:0/91810980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feca0109f90 0x7feca01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 --2- 172.21.15.96:0/91810980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109af0 0x7feca0101b70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 -- 172.21.15.96:0/91810980 >> 172.21.15.96:0/91810980 conn(0x7feca0077880 msgr2=0x7feca0077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:56.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 -- 172.21.15.96:0/91810980 shutdown_connections 2024-04-25T17:20:56.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 -- 172.21.15.96:0/91810980 wait complete. 2024-04-25T17:20:56.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 Processor -- start 2024-04-25T17:20:56.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.687+0000 7feca5f7d640 1 -- start start 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0102720 0x7feca019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feca0109af0 0x7feca019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109f90 0x7feca019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feca0113d00 con 0x7feca0109f90 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feca0113b80 con 0x7feca0102720 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feca0113e80 con 0x7feca0109af0 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109f90 0x7feca019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109f90 0x7feca019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40810/0 (socket says 172.21.15.96:40810) 2024-04-25T17:20:56.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 -- 172.21.15.96:0/2842181411 learned_addr learned my addr 172.21.15.96:0/2842181411 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9effd640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feca0109af0 0x7feca019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9f7fe640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0102720 0x7feca019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 -- 172.21.15.96:0/2842181411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feca0109af0 msgr2=0x7feca019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feca0109af0 0x7feca019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 -- 172.21.15.96:0/2842181411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0102720 msgr2=0x7feca019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0102720 0x7feca019a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 -- 172.21.15.96:0/2842181411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca019f5c0 con 0x7feca0109f90 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9effd640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feca0109af0 0x7feca019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9ffff640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109f90 0x7feca019ef80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fec9800dac0 tx=0x7fec9800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9cff9640 1 -- 172.21.15.96:0/2842181411 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98017070 con 0x7feca0109f90 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca019f8b0 con 0x7feca0109f90 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9cff9640 1 -- 172.21.15.96:0/2842181411 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9800be30 con 0x7feca0109f90 2024-04-25T17:20:56.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7fec9cff9640 1 -- 172.21.15.96:0/2842181411 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98010460 con 0x7feca0109f90 2024-04-25T17:20:56.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.688+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca019fdb0 con 0x7feca0109f90 2024-04-25T17:20:56.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.690+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca0104e90 con 0x7feca0109f90 2024-04-25T17:20:56.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.691+0000 7fec9cff9640 1 -- 172.21.15.96:0/2842181411 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7fec980040b0 con 0x7feca0109f90 2024-04-25T17:20:56.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.692+0000 7fec9cff9640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fec880afc60 0x7fec880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:56.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.692+0000 7fec9f7fe640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fec880afc60 0x7fec880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:56.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.692+0000 7fec9cff9640 1 -- 172.21.15.96:0/2842181411 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fec980dee40 con 0x7feca0109f90 2024-04-25T17:20:56.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.692+0000 7fec9f7fe640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fec880afc60 0x7fec880b2170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec90005f70 tx=0x7fec90000bc0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:56.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.695+0000 7fec9cff9640 1 -- 172.21.15.96:0/2842181411 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feca019fdb0 con 0x7feca0109f90 2024-04-25T17:20:56.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.874+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feca0108020 con 0x7feca0109f90 2024-04-25T17:20:56.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.875+0000 7fec9cff9640 1 -- 172.21.15.96:0/2842181411 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fec980a76d0 con 0x7feca0109f90 2024-04-25T17:20:56.876 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T17:20:56.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fec880afc60 msgr2=0x7fec880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fec880afc60 0x7fec880b2170 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec90005f70 tx=0x7fec90000bc0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109f90 msgr2=0x7feca019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109f90 0x7feca019ef80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fec9800dac0 tx=0x7fec9800df80 comp rx=0 tx=0).stop 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 shutdown_connections 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feca0109f90 0x7feca019ef80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feca0109af0 0x7feca019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7fec880afc60 0x7fec880b2170 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 --2- 172.21.15.96:0/2842181411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0102720 0x7feca019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 >> 172.21.15.96:0/2842181411 conn(0x7feca0077880 msgr2=0x7feca01051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 shutdown_connections 2024-04-25T17:20:56.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.877+0000 7feca5f7d640 1 -- 172.21.15.96:0/2842181411 wait complete. 2024-04-25T17:20:56.890 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T17:20:57.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 -- 172.21.15.96:0/4080481818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 msgr2=0x7f8360110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 --2- 172.21.15.96:0/4080481818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f8360110630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8358009ca0 tx=0x7f835802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 -- 172.21.15.96:0/4080481818 shutdown_connections 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 --2- 172.21.15.96:0/4080481818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f8360110630 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 --2- 172.21.15.96:0/4080481818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360109fb0 0x7f8360103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 --2- 172.21.15.96:0/4080481818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8360109b10 0x7f83601036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 -- 172.21.15.96:0/4080481818 >> 172.21.15.96:0/4080481818 conn(0x7f83600ff210 msgr2=0x7f8360101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 -- 172.21.15.96:0/4080481818 shutdown_connections 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.997+0000 7f8364892640 1 -- 172.21.15.96:0/4080481818 wait complete. 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 Processor -- start 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 -- start start 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f836019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8360109b10 0x7f836019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360109fb0 0x7f836019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8360113d20 con 0x7f8360104260 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8360113ba0 con 0x7f8360109fb0 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f8364892640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8360113ea0 con 0x7f8360109b10 2024-04-25T17:20:57.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.998+0000 7f835e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f836019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f836019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40834/0 (socket says 172.21.15.96:40834) 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 -- 172.21.15.96:0/1002413376 learned_addr learned my addr 172.21.15.96:0/1002413376 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835dd9b640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8360109b10 0x7f836019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835ed9d640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360109fb0 0x7f836019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 -- 172.21.15.96:0/1002413376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8360109b10 msgr2=0x7f836019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8360109b10 0x7f836019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 -- 172.21.15.96:0/1002413376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360109fb0 msgr2=0x7f836019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360109fb0 0x7f836019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 -- 172.21.15.96:0/1002413376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836019f560 con 0x7f8360104260 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f835e59c640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f836019a700 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f835000d740 tx=0x7f835000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f834f7fe640 1 -- 172.21.15.96:0/1002413376 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350017070 con 0x7f8360104260 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f834f7fe640 1 -- 172.21.15.96:0/1002413376 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f835000de50 con 0x7f8360104260 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836019f850 con 0x7f8360104260 2024-04-25T17:20:57.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f834f7fe640 1 -- 172.21.15.96:0/1002413376 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350004ce0 con 0x7f8360104260 2024-04-25T17:20:57.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:56.999+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836019fd50 con 0x7f8360104260 2024-04-25T17:20:57.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.000+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836006a160 con 0x7f8360104260 2024-04-25T17:20:57.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.001+0000 7f834f7fe640 1 -- 172.21.15.96:0/1002413376 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f8350004e80 con 0x7f8360104260 2024-04-25T17:20:57.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.002+0000 7f834f7fe640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83480afc60 0x7f83480b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.002+0000 7f835dd9b640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83480afc60 0x7f83480b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.002+0000 7f834f7fe640 1 -- 172.21.15.96:0/1002413376 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f83500dee00 con 0x7f8360104260 2024-04-25T17:20:57.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.003+0000 7f835dd9b640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83480afc60 0x7f83480b2170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8360108be0 tx=0x7f835400a2d0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:57.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.006+0000 7f834f7fe640 1 -- 172.21.15.96:0/1002413376 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f836019fd50 con 0x7f8360104260 2024-04-25T17:20:57.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.202+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f836010bc20 con 0x7f8360104260 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.203+0000 7f834f7fe640 1 -- 172.21.15.96:0/1002413376 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v88) ==== 81+0+942 (secure 0 0 0) 0x7f83500a7690 con 0x7f8360104260 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/559998201 2024-04-26T17:10:14.788578+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1944193211 2024-04-26T17:10:14.788578+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3419689135 2024-04-26T17:10:14.788578+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/3530143863 2024-04-26T17:10:14.788578+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/3530143863 2024-04-26T17:10:14.788578+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/4231768717 2024-04-26T17:01:02.956079+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/328848234 2024-04-26T17:10:14.788578+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/1771523969 2024-04-26T17:01:02.956079+0000 2024-04-25T17:20:57.204 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/1771523969 2024-04-26T17:01:02.956079+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2952856121 2024-04-26T17:01:02.956079+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/548635250 2024-04-26T17:00:35.573967+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3206171750 2024-04-26T17:00:35.573967+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/4227184783 2024-04-26T17:00:35.573967+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2705351311 2024-04-26T17:00:35.573967+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/4227184783 2024-04-26T17:00:35.573967+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/806329858 2024-04-26T17:01:02.956079+0000 2024-04-25T17:20:57.205 INFO:teuthology.orchestra.run.smithi096.stderr:listed 16 entries 2024-04-25T17:20:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83480afc60 msgr2=0x7f83480b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83480afc60 0x7f83480b2170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8360108be0 tx=0x7f835400a2d0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 msgr2=0x7f836019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f836019a700 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f835000d740 tx=0x7f835000dc00 comp rx=0 tx=0).stop 2024-04-25T17:20:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 shutdown_connections 2024-04-25T17:20:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f83480afc60 0x7f83480b2170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8360109fb0 0x7f836019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8360109b10 0x7f836019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 --2- 172.21.15.96:0/1002413376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8360104260 0x7f836019a700 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 >> 172.21.15.96:0/1002413376 conn(0x7f83600ff210 msgr2=0x7f83601018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:57.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 shutdown_connections 2024-04-25T17:20:57.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.205+0000 7f8364892640 1 -- 172.21.15.96:0/1002413376 wait complete. 2024-04-25T17:20:57.217 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T17:20:57.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:20:57.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T17:20:57.256 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:20:57.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:20:57.256 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T17:20:57.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:20:57.324 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T17:20:57.379 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T17:20:57.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:20:57.379 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T17:20:57.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:57 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/724891079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:57 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/370911225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:57 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/2842181411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T17:20:57.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-04-25T17:20:57.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:57 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/724891079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:57 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/370911225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:57 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/2842181411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:57 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/724891079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:57 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/370911225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:57 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/2842181411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T17:20:57.537 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T17:20:57.548 INFO:tasks.cephadm:Teardown begin 2024-04-25T17:20:57.548 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T17:20:57.548 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:20:57.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 -- 172.21.15.96:0/583583522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c108aa0 msgr2=0x7f814c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 --2- 172.21.15.96:0/583583522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c108aa0 0x7f814c102ca0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f813c009900 tx=0x7f813c02a410 comp rx=0 tx=0).stop 2024-04-25T17:20:57.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 -- 172.21.15.96:0/583583522 shutdown_connections 2024-04-25T17:20:57.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 --2- 172.21.15.96:0/583583522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f814c103850 0x7f814c10f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 --2- 172.21.15.96:0/583583522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c108f40 0x7f814c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 --2- 172.21.15.96:0/583583522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c108aa0 0x7f814c102ca0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 -- 172.21.15.96:0/583583522 >> 172.21.15.96:0/583583522 conn(0x7f814c0fd860 msgr2=0x7f814c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 -- 172.21.15.96:0/583583522 shutdown_connections 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 -- 172.21.15.96:0/583583522 wait complete. 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.657+0000 7f8150d7d640 1 Processor -- start 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8150d7d640 1 -- start start 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8150d7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c103850 0x7f814c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8150d7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c108aa0 0x7f814c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8150d7d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f814c108f40 0x7f814c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8150d7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f814c113db0 con 0x7f814c103850 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8150d7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f814c113c30 con 0x7f814c108aa0 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8150d7d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f814c113f30 con 0x7f814c108f40 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c103850 0x7f814c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c103850 0x7f814c19a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42204/0 (socket says 172.21.15.96:42204) 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 -- 172.21.15.96:0/3987584484 learned_addr learned my addr 172.21.15.96:0/3987584484 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f8143fff640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c108aa0 0x7f814c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814ad9d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f814c108f40 0x7f814c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 -- 172.21.15.96:0/3987584484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f814c108f40 msgr2=0x7f814c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f814c108f40 0x7f814c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 -- 172.21.15.96:0/3987584484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c108aa0 msgr2=0x7f814c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c108aa0 0x7f814c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 -- 172.21.15.96:0/3987584484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c19f630 con 0x7f814c103850 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814ad9d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f814c108f40 0x7f814c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.658+0000 7f814a59c640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c103850 0x7f814c19a7d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f813c032fd0 tx=0x7f813c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.659+0000 7f81437fe640 1 -- 172.21.15.96:0/3987584484 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c03f030 con 0x7f814c103850 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.659+0000 7f81437fe640 1 -- 172.21.15.96:0/3987584484 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f813c004490 con 0x7f814c103850 2024-04-25T17:20:57.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.659+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c19f8c0 con 0x7f814c103850 2024-04-25T17:20:57.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.659+0000 7f81437fe640 1 -- 172.21.15.96:0/3987584484 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c035070 con 0x7f814c103850 2024-04-25T17:20:57.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.659+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c19fe00 con 0x7f814c103850 2024-04-25T17:20:57.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.660+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c06a190 con 0x7f814c103850 2024-04-25T17:20:57.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.661+0000 7f81437fe640 1 -- 172.21.15.96:0/3987584484 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f813c042030 con 0x7f814c103850 2024-04-25T17:20:57.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.662+0000 7f81437fe640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f81380afce0 0x7f81380b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:57.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.662+0000 7f8143fff640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f81380afce0 0x7f81380b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:57.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.662+0000 7f81437fe640 1 -- 172.21.15.96:0/3987584484 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f813c0ff9d0 con 0x7f814c103850 2024-04-25T17:20:57.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.662+0000 7f8143fff640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f81380afce0 0x7f81380b21f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8134005fd0 tx=0x7f813400b510 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.666+0000 7f81437fe640 1 -- 172.21.15.96:0/3987584484 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f813c0c8260 con 0x7f814c103850 2024-04-25T17:20:57.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.882+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f814c1a7300 con 0x7f814c103850 2024-04-25T17:20:57.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.883+0000 7f81437fe640 1 -- 172.21.15.96:0/3987584484 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11808 (secure 0 0 0) 0x7f813c0c7b60 con 0x7f814c103850 2024-04-25T17:20:57.884 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:20:57.884 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:12:41.552227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25148":{"gid":25148,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3526163799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3526163799},{"type":"v1","addr":"172.21.15.97:6837","nonce":3526163799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:20:57.885 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 20 2024-04-25T17:20:57.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f81380afce0 msgr2=0x7f81380b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f81380afce0 0x7f81380b21f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8134005fd0 tx=0x7f813400b510 comp rx=0 tx=0).stop 2024-04-25T17:20:57.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c103850 msgr2=0x7f814c19a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:57.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c103850 0x7f814c19a7d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f813c032fd0 tx=0x7f813c004270 comp rx=0 tx=0).stop 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 shutdown_connections 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f814c108f40 0x7f814c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f81380afce0 0x7f81380b21f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f814c108aa0 0x7f814c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 --2- 172.21.15.96:0/3987584484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f814c103850 0x7f814c19a7d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 >> 172.21.15.96:0/3987584484 conn(0x7f814c0fd860 msgr2=0x7f814c0ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 shutdown_connections 2024-04-25T17:20:57.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:57.885+0000 7f8150d7d640 1 -- 172.21.15.96:0/3987584484 wait complete. 2024-04-25T17:20:57.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T17:20:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 -- 172.21.15.96:0/3626828805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 msgr2=0x7f5a0c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 --2- 172.21.15.96:0/3626828805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c102f00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009900 tx=0x7f59f402a410 comp rx=0 tx=0).stop 2024-04-25T17:20:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 -- 172.21.15.96:0/3626828805 shutdown_connections 2024-04-25T17:20:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 --2- 172.21.15.96:0/3626828805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c10d380 0x7f5a0c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 --2- 172.21.15.96:0/3626828805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c102f00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 --2- 172.21.15.96:0/3626828805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0c108a20 0x7f5a0c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 -- 172.21.15.96:0/3626828805 >> 172.21.15.96:0/3626828805 conn(0x7f5a0c078110 msgr2=0x7f5a0c078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 -- 172.21.15.96:0/3626828805 shutdown_connections 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.005+0000 7f5a11feb640 1 -- 172.21.15.96:0/3626828805 wait complete. 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 Processor -- start 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 -- start start 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c108a20 0x7f5a0c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0c10d380 0x7f5a0c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c112be0 con 0x7f5a0c102af0 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c112a60 con 0x7f5a0c108a20 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a11feb640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c112d60 con 0x7f5a0c10d380 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42222/0 (socket says 172.21.15.96:42222) 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 -- 172.21.15.96:0/460893820 learned_addr learned my addr 172.21.15.96:0/460893820 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0bfff640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0c10d380 0x7f5a0c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a03fff640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c108a20 0x7f5a0c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 -- 172.21.15.96:0/460893820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0c10d380 msgr2=0x7f5a0c19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0c10d380 0x7f5a0c19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 -- 172.21.15.96:0/460893820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c108a20 msgr2=0x7f5a0c199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c108a20 0x7f5a0c199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0b7fe640 1 -- 172.21.15.96:0/460893820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c19e400 con 0x7f5a0c102af0 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a0bfff640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0c10d380 0x7f5a0c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.006+0000 7f5a03fff640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c108a20 0x7f5a0c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.007+0000 7f5a0b7fe640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c199620 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00c9a0 tx=0x7f59fc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.007+0000 7f5a097fa640 1 -- 172.21.15.96:0/460893820 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc007bb0 con 0x7f5a0c102af0 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.007+0000 7f5a097fa640 1 -- 172.21.15.96:0/460893820 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59fc007d50 con 0x7f5a0c102af0 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.007+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c19e6f0 con 0x7f5a0c102af0 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.007+0000 7f5a097fa640 1 -- 172.21.15.96:0/460893820 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc018730 con 0x7f5a0c102af0 2024-04-25T17:20:58.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.007+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c19ebf0 con 0x7f5a0c102af0 2024-04-25T17:20:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.008+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c103cd0 con 0x7f5a0c102af0 2024-04-25T17:20:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.009+0000 7f5a097fa640 1 -- 172.21.15.96:0/460893820 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f59fc00f990 con 0x7f5a0c102af0 2024-04-25T17:20:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.010+0000 7f5a097fa640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f59f80afc60 0x7f59f80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.010+0000 7f5a03fff640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f59f80afc60 0x7f59f80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.010+0000 7f5a097fa640 1 -- 172.21.15.96:0/460893820 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f59fc0df540 con 0x7f5a0c102af0 2024-04-25T17:20:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.011+0000 7f5a03fff640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f59f80afc60 0x7f59f80b2170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59f4000c00 tx=0x7f59f4031400 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:58.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.014+0000 7f5a097fa640 1 -- 172.21.15.96:0/460893820 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59fc014030 con 0x7f5a0c102af0 2024-04-25T17:20:58.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.229+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a0c106f10 con 0x7f5a0c102af0 2024-04-25T17:20:58.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.230+0000 7f5a097fa640 1 -- 172.21.15.96:0/460893820 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11808 (secure 0 0 0) 0x7f59fc0a7dd0 con 0x7f5a0c102af0 2024-04-25T17:20:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:20:58.231 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2619590214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2619590214},{"type":"v1","addr":"172.21.15.96:6835","nonce":2619590214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2018971567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2018971567},{"type":"v1","addr":"172.21.15.96:6837","nonce":2018971567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14655,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/416971885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":416971885},{"type":"v1","addr":"172.21.15.96:6839","nonce":416971885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14667,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/1046285931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":1046285931},{"type":"v1","addr":"172.21.15.96:6841","nonce":1046285931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3848941544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3848941544},{"type":"v1","addr":"172.21.15.171:6833","nonce":3848941544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3336646389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3336646389},{"type":"v1","addr":"172.21.15.97:6833","nonce":3336646389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1302443885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1302443885},{"type":"v1","addr":"172.21.15.171:6835","nonce":1302443885}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1065540086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1065540086},{"type":"v1","addr":"172.21.15.171:6837","nonce":1065540086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24461,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3531324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3531324294},{"type":"v1","addr":"172.21.15.97:6835","nonce":3531324294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2758729388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2758729388},{"type":"v1","addr":"172.21.15.171:6839","nonce":2758729388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T17:08:11.494613+0000","modified":"2024-04-25T17:12:41.552227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3298672156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3298672156},{"type":"v1","addr":"172.21.15.97:6839","nonce":3298672156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25148":{"gid":25148,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3526163799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3526163799},{"type":"v1","addr":"172.21.15.97:6837","nonce":3526163799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T17:20:58.231 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 20 2024-04-25T17:20:58.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f59f80afc60 msgr2=0x7f59f80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f59f80afc60 0x7f59f80b2170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59f4000c00 tx=0x7f59f4031400 comp rx=0 tx=0).stop 2024-04-25T17:20:58.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 msgr2=0x7f5a0c199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c199620 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00c9a0 tx=0x7f59fc00ce60 comp rx=0 tx=0).stop 2024-04-25T17:20:58.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 shutdown_connections 2024-04-25T17:20:58.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a0c10d380 0x7f5a0c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f59f80afc60 0x7f59f80b2170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0c108a20 0x7f5a0c199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 --2- 172.21.15.96:0/460893820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a0c102af0 0x7f5a0c199620 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 >> 172.21.15.96:0/460893820 conn(0x7f5a0c078110 msgr2=0x7f5a0c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:58.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 shutdown_connections 2024-04-25T17:20:58.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.232+0000 7f5a11feb640 1 -- 172.21.15.96:0/460893820 wait complete. 2024-04-25T17:20:58.244 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T17:20:58.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:58 smithi171 ceph-mon[33189]: pgmap v333: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:58.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:58 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1002413376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T17:20:58.281 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:58 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3987584484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:20:58.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 -- 172.21.15.96:0/521550971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d901031e0 msgr2=0x7f7d9010f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 --2- 172.21.15.96:0/521550971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d901031e0 0x7f7d9010f4c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009ca0 tx=0x7f7d8002a8c0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 -- 172.21.15.96:0/521550971 shutdown_connections 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 --2- 172.21.15.96:0/521550971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d901031e0 0x7f7d9010f4c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 --2- 172.21.15.96:0/521550971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d90108ed0 0x7f7d90102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 --2- 172.21.15.96:0/521550971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d90108a30 0x7f7d90102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 -- 172.21.15.96:0/521550971 >> 172.21.15.96:0/521550971 conn(0x7f7d900781f0 msgr2=0x7f7d90078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 -- 172.21.15.96:0/521550971 shutdown_connections 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.351+0000 7f7d9592e640 1 -- 172.21.15.96:0/521550971 wait complete. 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 Processor -- start 2024-04-25T17:20:58.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 -- start start 2024-04-25T17:20:58.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d901031e0 0x7f7d9019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d90108a30 0x7f7d9019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d90108ed0 0x7f7d9019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90112c70 con 0x7f7d90108a30 2024-04-25T17:20:58.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90112af0 con 0x7f7d901031e0 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d9592e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d90112df0 con 0x7f7d90108ed0 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d90108a30 0x7f7d9019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d90108a30 0x7f7d9019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42230/0 (socket says 172.21.15.96:42230) 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 -- 172.21.15.96:0/1420843932 learned_addr learned my addr 172.21.15.96:0/1420843932 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8f7fe640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d90108ed0 0x7f7d9019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8effd640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d901031e0 0x7f7d9019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 -- 172.21.15.96:0/1420843932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d90108ed0 msgr2=0x7f7d9019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d90108ed0 0x7f7d9019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 -- 172.21.15.96:0/1420843932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d901031e0 msgr2=0x7f7d9019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d901031e0 0x7f7d9019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8e7fc640 1 -- 172.21.15.96:0/1420843932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9019f4e0 con 0x7f7d90108a30 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.352+0000 7f7d8effd640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d901031e0 0x7f7d9019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.353+0000 7f7d8e7fc640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d90108a30 0x7f7d9019abe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9f0 tx=0x7f7d7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:58.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.353+0000 7f7d9492c640 1 -- 172.21.15.96:0/1420843932 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800ccc0 con 0x7f7d90108a30 2024-04-25T17:20:58.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.353+0000 7f7d9492c640 1 -- 172.21.15.96:0/1420843932 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d780035f0 con 0x7f7d90108a30 2024-04-25T17:20:58.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.353+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9019f7d0 con 0x7f7d90108a30 2024-04-25T17:20:58.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.353+0000 7f7d9492c640 1 -- 172.21.15.96:0/1420843932 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800ce60 con 0x7f7d90108a30 2024-04-25T17:20:58.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.353+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d901a7150 con 0x7f7d90108a30 2024-04-25T17:20:58.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.354+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9006a190 con 0x7f7d90108a30 2024-04-25T17:20:58.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.355+0000 7f7d9492c640 1 -- 172.21.15.96:0/1420843932 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7d78003aa0 con 0x7f7d90108a30 2024-04-25T17:20:58.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.356+0000 7f7d9492c640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7d7c0afc60 0x7f7d7c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.356+0000 7f7d8effd640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7d7c0afc60 0x7f7d7c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.357+0000 7f7d9492c640 1 -- 172.21.15.96:0/1420843932 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f7d780df540 con 0x7f7d90108a30 2024-04-25T17:20:58.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.357+0000 7f7d8effd640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7d7c0afc60 0x7f7d7c0b2170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d90068f40 tx=0x7f7d88005e50 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:58.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.360+0000 7f7d9492c640 1 -- 172.21.15.96:0/1420843932 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d780a7dd0 con 0x7f7d90108a30 2024-04-25T17:20:58.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:58 smithi096 ceph-mon[29131]: pgmap v333: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:58.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:58 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1002413376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T17:20:58.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:58 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3987584484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:20:58.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-mon[32602]: pgmap v333: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T17:20:58.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1002413376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T17:20:58.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3987584484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:20:58.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.538+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d901a7440 con 0x7f7d90108a30 2024-04-25T17:20:58.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.540+0000 7f7d9492c640 1 -- 172.21.15.96:0/1420843932 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23060 (secure 0 0 0) 0x7f7d780a76d0 con 0x7f7d90108a30 2024-04-25T17:20:58.541 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T17:20:58.541 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"1be09d02-0325-11ef-bc93-c7b262605968","created":"2024-04-25T17:00:08.429242+0000","modified":"2024-04-25T17:20:18.541083+0000","last_up_change":"2024-04-25T17:07:21.441260+0000","last_in_change":"2024-04-25T17:07:00.732980+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T17:03:27.048575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T17:08:09.266201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T17:08:10.452964+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":87,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b0086d3f-f5b8-46e8-95db-03996e609f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6803","nonce":1323373413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6805","nonce":1323373413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6809","nonce":1323373413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1323373413},{"type":"v1","addr":"172.21.15.96:6807","nonce":1323373413}]},"public_addr":"172.21.15.96:6803/1323373413","cluster_addr":"172.21.15.96:6805/1323373413","heartbeat_back_addr":"172.21.15.96:6809/1323373413","heartbeat_front_addr":"172.21.15.96:6807/1323373413","state":["exists","up"]},{"osd":1,"uuid":"7e42e40f-3157-4e43-b9fc-689bacb722b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6801","nonce":1211116017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6803","nonce":1211116017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6807","nonce":1211116017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":1211116017},{"type":"v1","addr":"172.21.15.97:6805","nonce":1211116017}]},"public_addr":"172.21.15.97:6801/1211116017","cluster_addr":"172.21.15.97:6803/1211116017","heartbeat_back_addr":"172.21.15.97:6807/1211116017","heartbeat_front_addr":"172.21.15.97:6805/1211116017","state":["exists","up"]},{"osd":2,"uuid":"4a1d306b-692d-446a-a5af-40c55c06522e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6801","nonce":3917754473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6803","nonce":3917754473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6807","nonce":3917754473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":3917754473},{"type":"v1","addr":"172.21.15.171:6805","nonce":3917754473}]},"public_addr":"172.21.15.171:6801/3917754473","cluster_addr":"172.21.15.171:6803/3917754473","heartbeat_back_addr":"172.21.15.171:6807/3917754473","heartbeat_front_addr":"172.21.15.171:6805/3917754473","state":["exists","up"]},{"osd":3,"uuid":"abdaf538-404e-4714-b1a2-dcc2c22f275b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6811","nonce":3604096933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6813","nonce":3604096933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6817","nonce":3604096933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3604096933},{"type":"v1","addr":"172.21.15.96:6815","nonce":3604096933}]},"public_addr":"172.21.15.96:6811/3604096933","cluster_addr":"172.21.15.96:6813/3604096933","heartbeat_back_addr":"172.21.15.96:6817/3604096933","heartbeat_front_addr":"172.21.15.96:6815/3604096933","state":["exists","up"]},{"osd":4,"uuid":"27140752-dc84-4eb3-a875-5637f11a800a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6809","nonce":2529478189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6811","nonce":2529478189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6815","nonce":2529478189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2529478189},{"type":"v1","addr":"172.21.15.97:6813","nonce":2529478189}]},"public_addr":"172.21.15.97:6809/2529478189","cluster_addr":"172.21.15.97:6811/2529478189","heartbeat_back_addr":"172.21.15.97:6815/2529478189","heartbeat_front_addr":"172.21.15.97:6813/2529478189","state":["exists","up"]},{"osd":5,"uuid":"d623b6a2-12d2-4564-a836-e726059745f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6809","nonce":651283104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6811","nonce":651283104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6815","nonce":651283104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":651283104},{"type":"v1","addr":"172.21.15.171:6813","nonce":651283104}]},"public_addr":"172.21.15.171:6809/651283104","cluster_addr":"172.21.15.171:6811/651283104","heartbeat_back_addr":"172.21.15.171:6815/651283104","heartbeat_front_addr":"172.21.15.171:6813/651283104","state":["exists","up"]},{"osd":6,"uuid":"3f6a0ae2-7293-440b-9fcc-385526e69434","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6819","nonce":2945403945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6821","nonce":2945403945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6825","nonce":2945403945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2945403945},{"type":"v1","addr":"172.21.15.96:6823","nonce":2945403945}]},"public_addr":"172.21.15.96:6819/2945403945","cluster_addr":"172.21.15.96:6821/2945403945","heartbeat_back_addr":"172.21.15.96:6825/2945403945","heartbeat_front_addr":"172.21.15.96:6823/2945403945","state":["exists","up"]},{"osd":7,"uuid":"2281b753-0cf4-41de-b12c-9dda1afa5c71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6817","nonce":2873132021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6819","nonce":2873132021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6823","nonce":2873132021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2873132021},{"type":"v1","addr":"172.21.15.97:6821","nonce":2873132021}]},"public_addr":"172.21.15.97:6817/2873132021","cluster_addr":"172.21.15.97:6819/2873132021","heartbeat_back_addr":"172.21.15.97:6823/2873132021","heartbeat_front_addr":"172.21.15.97:6821/2873132021","state":["exists","up"]},{"osd":8,"uuid":"521eeba2-7a5c-44a5-839e-d57d391c2a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6817","nonce":2677669720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6819","nonce":2677669720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6823","nonce":2677669720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":2677669720},{"type":"v1","addr":"172.21.15.171:6821","nonce":2677669720}]},"public_addr":"172.21.15.171:6817/2677669720","cluster_addr":"172.21.15.171:6819/2677669720","heartbeat_back_addr":"172.21.15.171:6823/2677669720","heartbeat_front_addr":"172.21.15.171:6821/2677669720","state":["exists","up"]},{"osd":9,"uuid":"9d96f5d9-4be8-445e-a878-b9c296673d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6827","nonce":454601194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6829","nonce":454601194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6833","nonce":454601194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":454601194},{"type":"v1","addr":"172.21.15.96:6831","nonce":454601194}]},"public_addr":"172.21.15.96:6827/454601194","cluster_addr":"172.21.15.96:6829/454601194","heartbeat_back_addr":"172.21.15.96:6833/454601194","heartbeat_front_addr":"172.21.15.96:6831/454601194","state":["exists","up"]},{"osd":10,"uuid":"50d42cae-d4f6-4e59-b0c0-8678555c6b4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6825","nonce":213999228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6827","nonce":213999228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6831","nonce":213999228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":213999228},{"type":"v1","addr":"172.21.15.97:6829","nonce":213999228}]},"public_addr":"172.21.15.97:6825/213999228","cluster_addr":"172.21.15.97:6827/213999228","heartbeat_back_addr":"172.21.15.97:6831/213999228","heartbeat_front_addr":"172.21.15.97:6829/213999228","state":["exists","up"]},{"osd":11,"uuid":"fec892a1-61fb-43aa-96db-30f1a3e58416","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6825","nonce":2756841821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6827","nonce":2756841821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6831","nonce":2756841821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2756841821},{"type":"v1","addr":"172.21.15.171:6829","nonce":2756841821}]},"public_addr":"172.21.15.171:6825/2756841821","cluster_addr":"172.21.15.171:6827/2756841821","heartbeat_back_addr":"172.21.15.171:6831/2756841821","heartbeat_front_addr":"172.21.15.171:6829/2756841821","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:02:36.494457+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:01.027219+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:24.517183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:03:49.265858+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:14.975800+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:04:39.954963+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:06.160300+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:32.954588+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:05:58.420125+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:25.627957+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:06:52.883680+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T17:07:19.561770+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/559998201":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/1944193211":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/3419689135":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6801/3530143863":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6800/3530143863":"2024-04-26T17:10:14.788578+0000","172.21.15.96:0/4231768717":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/328848234":"2024-04-26T17:10:14.788578+0000","172.21.15.96:6801/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:6800/1771523969":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/2952856121":"2024-04-26T17:01:02.956079+0000","172.21.15.96:0/548635250":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/3206171750":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6800/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/2705351311":"2024-04-26T17:00:35.573967+0000","172.21.15.96:6801/4227184783":"2024-04-26T17:00:35.573967+0000","172.21.15.96:0/806329858":"2024-04-26T17:01:02.956079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7d7c0afc60 msgr2=0x7f7d7c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7d7c0afc60 0x7f7d7c0b2170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d90068f40 tx=0x7f7d88005e50 comp rx=0 tx=0).stop 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d90108a30 msgr2=0x7f7d9019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d90108a30 0x7f7d9019abe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9f0 tx=0x7f7d7800eeb0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 shutdown_connections 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d90108ed0 0x7f7d9019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d90108a30 0x7f7d9019abe0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7d7c0afc60 0x7f7d7c0b2170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 --2- 172.21.15.96:0/1420843932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d901031e0 0x7f7d9019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 >> 172.21.15.96:0/1420843932 conn(0x7f7d900781f0 msgr2=0x7f7d901066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:58.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 shutdown_connections 2024-04-25T17:20:58.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.542+0000 7f7d9592e640 1 -- 172.21.15.96:0/1420843932 wait complete. 2024-04-25T17:20:58.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 -- 172.21.15.96:0/2543409458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f627010e550 msgr2=0x7f6270110970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 --2- 172.21.15.96:0/2543409458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f627010e550 0x7f6270110970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6268009ca0 tx=0x7f626802a8c0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 -- 172.21.15.96:0/2543409458 shutdown_connections 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 --2- 172.21.15.96:0/2543409458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f627010e550 0x7f6270110970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 --2- 172.21.15.96:0/2543409458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6270102050 0x7f6270102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 --2- 172.21.15.96:0/2543409458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6270109ba0 0x7f6270101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 -- 172.21.15.96:0/2543409458 >> 172.21.15.96:0/2543409458 conn(0x7f6270077880 msgr2=0x7f6270077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 -- 172.21.15.96:0/2543409458 shutdown_connections 2024-04-25T17:20:58.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 -- 172.21.15.96:0/2543409458 wait complete. 2024-04-25T17:20:58.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 Processor -- start 2024-04-25T17:20:58.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.663+0000 7f6275f7d640 1 -- start start 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6275f7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6270102050 0x7f6270102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6275f7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6270109ba0 0x7f6270103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6275f7d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627010e550 0x7f6270103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6275f7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6270113d10 con 0x7f6270102050 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6275f7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6270113b90 con 0x7f6270109ba0 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6275f7d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6270113e90 con 0x7f627010e550 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6270102050 0x7f6270102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6270102050 0x7f6270102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42252/0 (socket says 172.21.15.96:42252) 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 -- 172.21.15.96:0/3547438972 learned_addr learned my addr 172.21.15.96:0/3547438972 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626ffff640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627010e550 0x7f6270103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6266ffd640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6270109ba0 0x7f6270103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 -- 172.21.15.96:0/3547438972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627010e550 msgr2=0x7f6270103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627010e550 0x7f6270103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 -- 172.21.15.96:0/3547438972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6270109ba0 msgr2=0x7f6270103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6270109ba0 0x7f6270103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 -- 172.21.15.96:0/3547438972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62701a8940 con 0x7f6270102050 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f6266ffd640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6270109ba0 0x7f6270103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626f7fe640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6270102050 0x7f6270102cf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f626000c970 tx=0x7f626000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.664+0000 7f626d7fa640 1 -- 172.21.15.96:0/3547438972 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260007c70 con 0x7f6270102050 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.665+0000 7f626d7fa640 1 -- 172.21.15.96:0/3547438972 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6260007e10 con 0x7f6270102050 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.665+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62701a8c30 con 0x7f6270102050 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.665+0000 7f626d7fa640 1 -- 172.21.15.96:0/3547438972 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62600187b0 con 0x7f6270102050 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.665+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701a9180 con 0x7f6270102050 2024-04-25T17:20:58.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.665+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62700692e0 con 0x7f6270102050 2024-04-25T17:20:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.667+0000 7f626d7fa640 1 -- 172.21.15.96:0/3547438972 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f626000fab0 con 0x7f6270102050 2024-04-25T17:20:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.667+0000 7f626d7fa640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f625c0afc60 0x7f625c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.668+0000 7f6266ffd640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f625c0afc60 0x7f625c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.668+0000 7f626d7fa640 1 -- 172.21.15.96:0/3547438972 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f62600df2c0 con 0x7f6270102050 2024-04-25T17:20:58.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.668+0000 7f6266ffd640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f625c0afc60 0x7f625c0b2170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62701013d0 tx=0x7f625800a5c0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:58.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.671+0000 7f626d7fa640 1 -- 172.21.15.96:0/3547438972 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6260053ff0 con 0x7f6270102050 2024-04-25T17:20:58.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.887+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6270104220 con 0x7f6270102050 2024-04-25T17:20:58.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.890+0000 7f626d7fa640 1 -- 172.21.15.96:0/3547438972 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f6260014030 con 0x7f6270102050 2024-04-25T17:20:58.891 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f625c0afc60 msgr2=0x7f625c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f625c0afc60 0x7f625c0b2170 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62701013d0 tx=0x7f625800a5c0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6270102050 msgr2=0x7f6270102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6270102050 0x7f6270102cf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f626000c970 tx=0x7f626000ce30 comp rx=0 tx=0).stop 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 shutdown_connections 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f627010e550 0x7f6270103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f625c0afc60 0x7f625c0b2170 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6270109ba0 0x7f6270103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 --2- 172.21.15.96:0/3547438972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6270102050 0x7f6270102cf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 >> 172.21.15.96:0/3547438972 conn(0x7f6270077880 msgr2=0x7f627010a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:58.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 shutdown_connections 2024-04-25T17:20:58.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:58.893+0000 7f6275f7d640 1 -- 172.21.15.96:0/3547438972 wait complete. 2024-04-25T17:20:58.905 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T17:20:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 -- 172.21.15.96:0/1244913141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4102420 msgr2=0x7f84c4102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 --2- 172.21.15.96:0/1244913141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4102420 0x7f84c4102830 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009900 tx=0x7f84b402a410 comp rx=0 tx=0).stop 2024-04-25T17:20:59.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 -- 172.21.15.96:0/1244913141 shutdown_connections 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 --2- 172.21.15.96:0/1244913141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84c4107470 0x7f84c410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 --2- 172.21.15.96:0/1244913141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c41069b0 0x7f84c4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 --2- 172.21.15.96:0/1244913141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4102420 0x7f84c4102830 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 -- 172.21.15.96:0/1244913141 >> 172.21.15.96:0/1244913141 conn(0x7f84c4077920 msgr2=0x7f84c4077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 -- 172.21.15.96:0/1244913141 shutdown_connections 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 -- 172.21.15.96:0/1244913141 wait complete. 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 Processor -- start 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.017+0000 7f84c94f9640 1 -- start start 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c94f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c4102420 0x7f84c4110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c94f9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84c41069b0 0x7f84c4110bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c94f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4107470 0x7f84c4111110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:59.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c94f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4113d10 con 0x7f84c4107470 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c94f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4113b90 con 0x7f84c4102420 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c94f9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4113e90 con 0x7f84c41069b0 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4107470 0x7f84c4111110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4107470 0x7f84c4111110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42268/0 (socket says 172.21.15.96:42268) 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 -- 172.21.15.96:0/3695601557 learned_addr learned my addr 172.21.15.96:0/3695601557 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c2ffd640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c4102420 0x7f84c4110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84bbfff640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84c41069b0 0x7f84c4110bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 -- 172.21.15.96:0/3695601557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84c41069b0 msgr2=0x7f84c4110bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84c41069b0 0x7f84c4110bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 -- 172.21.15.96:0/3695601557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c4102420 msgr2=0x7f84c4110690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c4102420 0x7f84c4110690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 -- 172.21.15.96:0/3695601557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c41a8ad0 con 0x7f84c4107470 2024-04-25T17:20:59.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84bbfff640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84c41069b0 0x7f84c4110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T17:20:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c37fe640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4107470 0x7f84c4111110 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00eb10 tx=0x7f84bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c0ff9640 1 -- 172.21.15.96:0/3695601557 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc004b20 con 0x7f84c4107470 2024-04-25T17:20:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c0ff9640 1 -- 172.21.15.96:0/3695601557 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84bc004cc0 con 0x7f84c4107470 2024-04-25T17:20:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c41a8dc0 con 0x7f84c4107470 2024-04-25T17:20:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.018+0000 7f84c0ff9640 1 -- 172.21.15.96:0/3695601557 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc010670 con 0x7f84c4107470 2024-04-25T17:20:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.019+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c41a9280 con 0x7f84c4107470 2024-04-25T17:20:59.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.019+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c406a160 con 0x7f84c4107470 2024-04-25T17:20:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.021+0000 7f84c0ff9640 1 -- 172.21.15.96:0/3695601557 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f84bc005000 con 0x7f84c4107470 2024-04-25T17:20:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.022+0000 7f84c0ff9640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84b00afc60 0x7f84b00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:20:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.022+0000 7f84c2ffd640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84b00afc60 0x7f84b00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:20:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.022+0000 7f84c0ff9640 1 -- 172.21.15.96:0/3695601557 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8584+0+0 (secure 0 0 0) 0x7f84bc0df820 con 0x7f84c4107470 2024-04-25T17:20:59.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.022+0000 7f84c2ffd640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84b00afc60 0x7f84b00b2170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84b4031440 tx=0x7f84b4009560 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:20:59.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.025+0000 7f84c0ff9640 1 -- 172.21.15.96:0/3695601557 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84bc053ff0 con 0x7f84c4107470 2024-04-25T17:20:59.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a[29127]: 2024-04-25T17:20:58.888+0000 7fb0bf6d3700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T17:20:59.235 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b[62836]: ignoring --setuser ceph since I am not root 2024-04-25T17:20:59.235 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b[62836]: ignoring --setgroup ceph since I am not root 2024-04-25T17:20:59.235 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-mds[62840]: -- 172.21.15.97:0/773631015 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fd5f71a40 con 0x555fd5fb9a80 2024-04-25T17:20:59.235 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b[62836]: starting mds.b at 2024-04-25T17:20:59.235 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[66208]: ignoring --setuser ceph since I am not root 2024-04-25T17:20:59.235 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[66208]: ignoring --setgroup ceph since I am not root 2024-04-25T17:20:59.236 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-mds[66212]: -- 172.21.15.97:0/582668279 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565f60a7a40 con 0x5565f60ef600 2024-04-25T17:20:59.236 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:20:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[66208]: starting mds.h at 2024-04-25T17:20:59.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/460893820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:20:59.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1420843932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:59.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3547438972' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T17:20:59.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T17:20:59.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T17:20:59.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3547438972' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T17:20:59.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T17:20:59.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:20:59 smithi097 ceph-mon[32602]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T17:20:59.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.243+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f84c41a9610 con 0x7f84c4107470 2024-04-25T17:20:59.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/460893820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:20:59.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1420843932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:59.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3547438972' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T17:20:59.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T17:20:59.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T17:20:59.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3547438972' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T17:20:59.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T17:20:59.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:20:59 smithi096 ceph-mon[29131]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T17:20:59.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/460893820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T17:20:59.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1420843932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T17:20:59.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3547438972' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T17:20:59.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T17:20:59.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T17:20:59.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3547438972' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T17:20:59.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T17:20:59.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:20:59 smithi171 ceph-mon[33189]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T17:20:59.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.895+0000 7f84c0ff9640 1 -- 172.21.15.96:0/3695601557 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f84bc0a8080 con 0x7f84c4107470 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84b00afc60 msgr2=0x7f84b00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84b00afc60 0x7f84b00b2170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84b4031440 tx=0x7f84b4009560 comp rx=0 tx=0).stop 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4107470 msgr2=0x7f84c4111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4107470 0x7f84c4111110 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00eb10 tx=0x7f84bc00c2d0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 shutdown_connections 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84c4107470 0x7f84c4111110 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84c41069b0 0x7f84c4110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f84b00afc60 0x7f84b00b2170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 --2- 172.21.15.96:0/3695601557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84c4102420 0x7f84c4110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:20:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 >> 172.21.15.96:0/3695601557 conn(0x7f84c4077920 msgr2=0x7f84c41008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:20:59.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 shutdown_connections 2024-04-25T17:20:59.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:20:59.898+0000 7f84c94f9640 1 -- 172.21.15.96:0/3695601557 wait complete. 2024-04-25T17:20:59.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T17:21:00.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 -- 172.21.15.96:0/2754412459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 msgr2=0x7f5a581025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:00.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 --2- 172.21.15.96:0/2754412459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a581025c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009900 tx=0x7f5a4802a410 comp rx=0 tx=0).stop 2024-04-25T17:21:00.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 -- 172.21.15.96:0/2754412459 shutdown_connections 2024-04-25T17:21:00.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 --2- 172.21.15.96:0/2754412459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a5810d390 0x7f5a5810f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:00.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 --2- 172.21.15.96:0/2754412459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a58102b00 0x7f5a58102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:00.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 --2- 172.21.15.96:0/2754412459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a581025c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:00.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 -- 172.21.15.96:0/2754412459 >> 172.21.15.96:0/2754412459 conn(0x7f5a580fe260 msgr2=0x7f5a58100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:21:00.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 -- 172.21.15.96:0/2754412459 shutdown_connections 2024-04-25T17:21:00.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.029+0000 7f5a5d364640 1 -- 172.21.15.96:0/2754412459 wait complete. 2024-04-25T17:21:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 Processor -- start 2024-04-25T17:21:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 -- start start 2024-04-25T17:21:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a58102b00 0x7f5a58199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a58199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a5810d390 0x7f5a5819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58112bf0 con 0x7f5a58108a30 2024-04-25T17:21:00.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58112a70 con 0x7f5a58102b00 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a5d364640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58112d70 con 0x7f5a5810d390 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.030+0000 7f5a567fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a58199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a58199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42288/0 (socket says 172.21.15.96:42288) 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 -- 172.21.15.96:0/1064864496 learned_addr learned my addr 172.21.15.96:0/1064864496 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a577fe640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a5810d390 0x7f5a5819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a56ffd640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a58102b00 0x7f5a58199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 -- 172.21.15.96:0/1064864496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a5810d390 msgr2=0x7f5a5819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a5810d390 0x7f5a5819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 -- 172.21.15.96:0/1064864496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a58102b00 msgr2=0x7f5a58199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a58102b00 0x7f5a58199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 -- 172.21.15.96:0/1064864496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5819e4d0 con 0x7f5a58108a30 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a567fc640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a58199bd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005d00 tx=0x7f5a4c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a3ffff640 1 -- 172.21.15.96:0/1064864496 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c017070 con 0x7f5a58108a30 2024-04-25T17:21:00.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5819e7c0 con 0x7f5a58108a30 2024-04-25T17:21:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a3ffff640 1 -- 172.21.15.96:0/1064864496 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a4c009c30 con 0x7f5a58108a30 2024-04-25T17:21:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a3ffff640 1 -- 172.21.15.96:0/1064864496 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00f620 con 0x7f5a58108a30 2024-04-25T17:21:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.031+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5819ecc0 con 0x7f5a58108a30 2024-04-25T17:21:00.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.032+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a58103ce0 con 0x7f5a58108a30 2024-04-25T17:21:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.034+0000 7f5a3ffff640 1 -- 172.21.15.96:0/1064864496 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f5a4c01a070 con 0x7f5a58108a30 2024-04-25T17:21:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.035+0000 7f5a3ffff640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f5a400afc60 0x7f5a400b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.035+0000 7f5a56ffd640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f5a400afc60 0x7f5a400b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.035+0000 7f5a3ffff640 1 -- 172.21.15.96:0/1064864496 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8584+0+0 (secure 0 0 0) 0x7f5a4c0deab0 con 0x7f5a58108a30 2024-04-25T17:21:00.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.036+0000 7f5a56ffd640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f5a400afc60 0x7f5a400b2170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a48000c00 tx=0x7f5a48031400 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:21:00.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.039+0000 7f5a3ffff640 1 -- 172.21.15.96:0/1064864496 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a5819ecc0 con 0x7f5a58108a30 2024-04-25T17:21:00.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:00.227+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5a5806a160 con 0x7f5a58108a30 2024-04-25T17:21:00.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: pgmap v335: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:21:00.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3695601557' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:21:00.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T17:21:00.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T17:21:00.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: Cluster is now healthy 2024-04-25T17:21:00.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/3695601557' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T17:21:00.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6836/3946935229,v1:172.21.15.97:6838/3946935229] up:boot 2024-04-25T17:21:00.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: mds.? [v2:172.21.15.97:6837/2476024127,v1:172.21.15.97:6839/2476024127] up:boot 2024-04-25T17:21:00.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: fsmap 12 up:standby 2024-04-25T17:21:00.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:21:00.440 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:00 smithi096 ceph-mon[29131]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: pgmap v335: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3695601557' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: Cluster is now healthy 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/3695601557' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6836/3946935229,v1:172.21.15.97:6838/3946935229] up:boot 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: mds.? [v2:172.21.15.97:6837/2476024127,v1:172.21.15.97:6839/2476024127] up:boot 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: fsmap 12 up:standby 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:21:00.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:00 smithi097 ceph-mon[32602]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: pgmap v335: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3695601557' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: Cluster is now healthy 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/3695601557' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6836/3946935229,v1:172.21.15.97:6838/3946935229] up:boot 2024-04-25T17:21:00.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: mds.? [v2:172.21.15.97:6837/2476024127,v1:172.21.15.97:6839/2476024127] up:boot 2024-04-25T17:21:00.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: fsmap 12 up:standby 2024-04-25T17:21:00.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T17:21:00.532 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:00 smithi171 ceph-mon[33189]: from='mgr.15261 172.21.15.96:0/1163230648' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T17:21:01.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.195+0000 7f5a3ffff640 1 -- 172.21.15.96:0/1064864496 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7f5a4c009de0 con 0x7f5a58108a30 2024-04-25T17:21:01.196 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.197+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f5a400afc60 msgr2=0x7f5a400b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.197+0000 7f5a5d364640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f5a400afc60 0x7f5a400b2170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a48000c00 tx=0x7f5a48031400 comp rx=0 tx=0).stop 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 msgr2=0x7f5a58199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a58199bd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005d00 tx=0x7f5a4c00dde0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 shutdown_connections 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a5810d390 0x7f5a5819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a58108a30 0x7f5a58199bd0 secure :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005d00 tx=0x7f5a4c00dde0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f5a400afc60 0x7f5a400b2170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 --2- 172.21.15.96:0/1064864496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a58102b00 0x7f5a58199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 >> 172.21.15.96:0/1064864496 conn(0x7f5a580fe260 msgr2=0x7f5a580fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:21:01.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 shutdown_connections 2024-04-25T17:21:01.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.198+0000 7f5a5d364640 1 -- 172.21.15.96:0/1064864496 wait complete. 2024-04-25T17:21:01.211 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T17:21:01.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 -- 172.21.15.96:0/3283858226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c10d390 msgr2=0x7f7f9c10f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:01.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/3283858226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c10d390 0x7f7f9c10f7b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009900 tx=0x7f7f9402a410 comp rx=0 tx=0).stop 2024-04-25T17:21:01.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 -- 172.21.15.96:0/3283858226 shutdown_connections 2024-04-25T17:21:01.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/3283858226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c10d390 0x7f7f9c10f7b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/3283858226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c102b00 0x7f7f9c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/3283858226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f9c108a30 0x7f7f9c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 -- 172.21.15.96:0/3283858226 >> 172.21.15.96:0/3283858226 conn(0x7f7f9c0fe080 msgr2=0x7f7f9c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 -- 172.21.15.96:0/3283858226 shutdown_connections 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.323+0000 7f7fa41f3640 1 -- 172.21.15.96:0/3283858226 wait complete. 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 Processor -- start 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 -- start start 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c102b00 0x7f7f9c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c108a30 0x7f7f9c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f9c10d390 0x7f7f9c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c112c70 con 0x7f7f9c108a30 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c112af0 con 0x7f7f9c102b00 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa41f3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c112df0 con 0x7f7f9c10d390 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa178e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c108a30 0x7f7f9c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:01.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa178e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c108a30 0x7f7f9c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42308/0 (socket says 172.21.15.96:42308) 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.324+0000 7f7fa178e640 1 -- 172.21.15.96:0/1214316978 learned_addr learned my addr 172.21.15.96:0/1214316978 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa178e640 1 -- 172.21.15.96:0/1214316978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f9c10d390 msgr2=0x7f7f9c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa2790640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f9c10d390 0x7f7f9c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa1f8f640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c102b00 0x7f7f9c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa178e640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f9c10d390 0x7f7f9c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa178e640 1 -- 172.21.15.96:0/1214316978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c102b00 msgr2=0x7f7f9c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa178e640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c102b00 0x7f7f9c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa178e640 1 -- 172.21.15.96:0/1214316978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9c19f4c0 con 0x7f7f9c108a30 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa1f8f640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c102b00 0x7f7f9c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa178e640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c108a30 0x7f7f9c19ac40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c9a0 tx=0x7f7f8c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7f92ffd640 1 -- 172.21.15.96:0/1214316978 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c007c70 con 0x7f7f9c108a30 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7f92ffd640 1 -- 172.21.15.96:0/1214316978 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8c007e10 con 0x7f7f9c108a30 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9c19f7b0 con 0x7f7f9c108a30 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7f92ffd640 1 -- 172.21.15.96:0/1214316978 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c018850 con 0x7f7f9c108a30 2024-04-25T17:21:01.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.325+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f9c1a71b0 con 0x7f7f9c108a30 2024-04-25T17:21:01.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.326+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f9c06a190 con 0x7f7f9c108a30 2024-04-25T17:21:01.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.327+0000 7f7f92ffd640 1 -- 172.21.15.96:0/1214316978 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 147651+0+0 (secure 0 0 0) 0x7f7f8c00fb40 con 0x7f7f9c108a30 2024-04-25T17:21:01.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.328+0000 7f7f92ffd640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7f880afc60 0x7f7f880b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T17:21:01.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.328+0000 7f7fa1f8f640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7f880afc60 0x7f7f880b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T17:21:01.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.328+0000 7f7f92ffd640 1 -- 172.21.15.96:0/1214316978 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7972+0+0 (secure 0 0 0) 0x7f7f8c0df0c0 con 0x7f7f9c108a30 2024-04-25T17:21:01.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.329+0000 7f7fa1f8f640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7f880afc60 0x7f7f880b2170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c103a30 tx=0x7f7f980073c0 comp rx=0 tx=0).ready entity=mgr.15261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T17:21:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.332+0000 7f7f92ffd640 1 -- 172.21.15.96:0/1214316978 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f8c014030 con 0x7f7f9c108a30 2024-04-25T17:21:01.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:01 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1064864496' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T17:21:01.439 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:21:01.345+0000 7f46b5715700 -1 cephsqlite: Access: (client.15294) cannot open striper 2024-04-25T17:21:01.439 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:21:01.345+0000 7f46b5715700 -1 client.15294: SimpleRADOSStriper: unlock: snap_db_v0.db: unlock failed: (2) No such file or directory 2024-04-25T17:21:01.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:01 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1064864496' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T17:21:01.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:01.517+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f9c19ba30 con 0x7f7f9c108a30 2024-04-25T17:21:01.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:01 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1064864496' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T17:21:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.196+0000 7f7f92ffd640 1 -- 172.21.15.96:0/1214316978 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f7f8c0a7c60 con 0x7f7f9c108a30 2024-04-25T17:21:02.197 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-04-25T17:21:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.199+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7f880afc60 msgr2=0x7f7f880b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.199+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7f880afc60 0x7f7f880b2170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c103a30 tx=0x7f7f980073c0 comp rx=0 tx=0).stop 2024-04-25T17:21:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c108a30 msgr2=0x7f7f9c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T17:21:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c108a30 0x7f7f9c19ac40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c9a0 tx=0x7f7f8c00ce60 comp rx=0 tx=0).stop 2024-04-25T17:21:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 shutdown_connections 2024-04-25T17:21:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f9c10d390 0x7f7f9c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:02.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f9c108a30 0x7f7f9c19ac40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:02.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.96:6800/1594342167,v1:172.21.15.96:6801/1594342167] conn(0x7f7f880afc60 0x7f7f880b2170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:02.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 --2- 172.21.15.96:0/1214316978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c102b00 0x7f7f9c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T17:21:02.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 >> 172.21.15.96:0/1214316978 conn(0x7f7f9c0fe080 msgr2=0x7f7f9c10e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T17:21:02.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 shutdown_connections 2024-04-25T17:21:02.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T17:21:02.200+0000 7f7fa41f3640 1 -- 172.21.15.96:0/1214316978 wait complete. 2024-04-25T17:21:02.222 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:21:02.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:21:02.286 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:21:02.316 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T17:21:02.316 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T17:21:02.331 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T17:21:02.349 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T17:21:02.372 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T17:21:02.373 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T17:21:02.373 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a 2024-04-25T17:21:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:02 smithi097 ceph-mon[32602]: pgmap v336: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 307 B/s wr, 3 op/s 2024-04-25T17:21:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:02 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1064864496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T17:21:02.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:02 smithi097 ceph-mon[32602]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T17:21:02.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:02 smithi097 ceph-mon[32602]: from='client.? 172.21.15.96:0/1214316978' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T17:21:02.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:02 smithi171 ceph-mon[33189]: pgmap v336: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 307 B/s wr, 3 op/s 2024-04-25T17:21:02.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:02 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1064864496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T17:21:02.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:02 smithi171 ceph-mon[33189]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T17:21:02.531 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:02 smithi171 ceph-mon[33189]: from='client.? 172.21.15.96:0/1214316978' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T17:21:02.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 ceph-mon[29131]: pgmap v336: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 307 B/s wr, 3 op/s 2024-04-25T17:21:02.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1064864496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T17:21:02.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 ceph-mon[29131]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T17:21:02.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 ceph-mon[29131]: from='client.? 172.21.15.96:0/1214316978' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T17:21:02.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 systemd[1]: Stopping Ceph mon.a for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:02.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a[29127]: 2024-04-25T17:21:02.676+0000 7fb0c4ede700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:02.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a[29127]: 2024-04-25T17:21:02.676+0000 7fb0c4ede700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T17:21:02.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:02 smithi096 podman[106307]: 2024-04-25 17:21:02.708582193 +0000 UTC m=+0.157575442 container died 08b03381202958ddfe15893731e59e5c00190c27de01501ad6919006009b6a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T17:21:03.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:03 smithi096 podman[106307]: 2024-04-25 17:21:03.086751639 +0000 UTC m=+0.535744883 container cleanup 08b03381202958ddfe15893731e59e5c00190c27de01501ad6919006009b6a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T17:21:03.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 17:21:03 smithi096 bash[106307]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-a 2024-04-25T17:21:03.504 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T17:21:03.548 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:03.549 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T17:21:03.549 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T17:21:03.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.b 2024-04-25T17:21:03.657 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:03 smithi097 systemd[1]: Stopping Ceph mon.b for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:03.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-b[32598]: 2024-04-25T17:21:03.880+0000 7f9b11b78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:03.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-b[32598]: 2024-04-25T17:21:03.880+0000 7f9b11b78700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T17:21:03.985 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:03 smithi097 podman[68499]: 2024-04-25 17:21:03.901747852 +0000 UTC m=+0.142733473 container died 1ea0d8872f77161079620a243b461b1b1c8d8b018de82e797d3f93426f38d708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T17:21:04.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:04 smithi097 podman[68499]: 2024-04-25 17:21:04.289339451 +0000 UTC m=+0.530325080 container cleanup 1ea0d8872f77161079620a243b461b1b1c8d8b018de82e797d3f93426f38d708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:21:04.485 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 17:21:04 smithi097 bash[68499]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-b 2024-04-25T17:21:04.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T17:21:04.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:04.773 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T17:21:04.773 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T17:21:04.773 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.c 2024-04-25T17:21:05.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:04 smithi171 systemd[1]: Stopping Ceph mon.c for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:05.189 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:04 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:21:04.931+0000 7f46ca27e700 -1 client.15294: SimpleRADOSStriper: lock_keeper_main: snap_db_v0.db: lock renewal failed: (2) No such file or directory 2024-04-25T17:21:05.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c[33182]: 2024-04-25T17:21:05.077+0000 7f5f34d61700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:05.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c[33182]: 2024-04-25T17:21:05.077+0000 7f5f34d61700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T17:21:05.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:05 smithi171 podman[70317]: 2024-04-25 17:21:05.169546684 +0000 UTC m=+0.206530440 container died 005b74a688e831b7a55d7592ad95d1cbb9665b34d9bc3985aa7423f4f9f9a483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:21:05.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:05 smithi171 podman[70317]: 2024-04-25 17:21:05.562252822 +0000 UTC m=+0.599236579 container cleanup 005b74a688e831b7a55d7592ad95d1cbb9665b34d9bc3985aa7423f4f9f9a483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=) 2024-04-25T17:21:05.781 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 25 17:21:05 smithi171 bash[70317]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mon-c 2024-04-25T17:21:05.982 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T17:21:06.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:06.030 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T17:21:06.030 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T17:21:06.030 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.x 2024-04-25T17:21:06.408 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:06 smithi096 systemd[1]: Stopping Ceph mgr.x for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:06.690 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:06 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:21:06.406+0000 7f47510eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:06.690 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:06 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x[29361]: 2024-04-25T17:21:06.406+0000 7f47510eb700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T17:21:06.690 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:06 smithi096 podman[106428]: 2024-04-25 17:21:06.438630051 +0000 UTC m=+0.221747827 container died a92cedda03dffebef043e87f9f232360ef1e6122b009f3621c7e11bba0f4223e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T17:21:07.169 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:06 smithi096 podman[106428]: 2024-04-25 17:21:06.942445883 +0000 UTC m=+0.725563658 container cleanup a92cedda03dffebef043e87f9f232360ef1e6122b009f3621c7e11bba0f4223e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T17:21:07.169 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 17:21:06 smithi096 bash[106428]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-x 2024-04-25T17:21:07.309 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T17:21:07.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:07.343 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T17:21:07.343 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T17:21:07.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.y 2024-04-25T17:21:07.683 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:21:07 smithi097 systemd[1]: Stopping Ceph mgr.y for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:07.979 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:21:07 smithi097 podman[68630]: 2024-04-25 17:21:07.683534275 +0000 UTC m=+0.152338499 container died fb87bae1086983a38364ad3451fac38709267693a779a65fe2a4e5ef475cefa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:08.234 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:21:08 smithi097 podman[68630]: 2024-04-25 17:21:08.078594637 +0000 UTC m=+0.547398861 container cleanup fb87bae1086983a38364ad3451fac38709267693a779a65fe2a4e5ef475cefa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T17:21:08.235 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 17:21:08 smithi097 bash[68630]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-y 2024-04-25T17:21:08.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T17:21:08.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:08.550 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T17:21:08.550 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T17:21:08.550 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.z 2024-04-25T17:21:08.890 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:21:08 smithi171 systemd[1]: Stopping Ceph mgr.z for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:09.212 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:21:08 smithi171 podman[70439]: 2024-04-25 17:21:08.890287523 +0000 UTC m=+0.141754710 container died 8202b0e3e79744967ccd1335d15d7770b79ca0a4f284dddf5c81f83d2fb0102a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T17:21:09.528 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:21:09 smithi171 podman[70439]: 2024-04-25 17:21:09.28542992 +0000 UTC m=+0.536897107 container cleanup 8202b0e3e79744967ccd1335d15d7770b79ca0a4f284dddf5c81f83d2fb0102a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:09.528 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 25 17:21:09 smithi171 bash[70439]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mgr-z 2024-04-25T17:21:09.650 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T17:21:09.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:09.684 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T17:21:09.684 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T17:21:09.685 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.0 2024-04-25T17:21:09.997 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:09 smithi096 systemd[1]: Stopping Ceph osd.0 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:10.439 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0[43637]: 2024-04-25T17:21:09.996+0000 7fc3062ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:10.439 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0[43637]: 2024-04-25T17:21:09.996+0000 7fc3062ed700 -1 osd.0 91 *** Got signal Terminated *** 2024-04-25T17:21:10.439 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0[43637]: 2024-04-25T17:21:09.996+0000 7fc3062ed700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:21:15.848 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:15 smithi096 podman[106553]: 2024-04-25 17:21:15.555286777 +0000 UTC m=+5.684234297 container died ee49bd16414ae9c80981e600976560f9f4c2edf1fda7fdb63e97181303924a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T17:21:16.173 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:15 smithi096 podman[106553]: 2024-04-25 17:21:15.938974149 +0000 UTC m=+6.067921675 container cleanup ee49bd16414ae9c80981e600976560f9f4c2edf1fda7fdb63e97181303924a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T17:21:16.173 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:15 smithi096 bash[106553]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0 2024-04-25T17:21:16.439 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:16 smithi096 podman[106572]: 2024-04-25 17:21:16.173822658 +0000 UTC m=+0.620095552 container remove ee49bd16414ae9c80981e600976560f9f4c2edf1fda7fdb63e97181303924a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T17:21:16.719 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:16 smithi096 podman[106664]: 2024-04-25 17:21:16.369453915 +0000 UTC m=+0.018032895 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:21:16.719 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:16 smithi096 podman[106664]: 2024-04-25 17:21:16.491445767 +0000 UTC m=+0.140024720 container create e740819b5e9fda0fed13dd357d14dba68853e19836c638f0120d9a743bb18f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:16.719 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:16 smithi096 podman[106664]: 2024-04-25 17:21:16.718929055 +0000 UTC m=+0.367508011 container init e740819b5e9fda0fed13dd357d14dba68853e19836c638f0120d9a743bb18f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T17:21:16.969 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:16 smithi096 podman[106664]: 2024-04-25 17:21:16.722433729 +0000 UTC m=+0.371012688 container start e740819b5e9fda0fed13dd357d14dba68853e19836c638f0120d9a743bb18f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T17:21:17.381 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:16 smithi096 podman[106664]: 2024-04-25 17:21:16.969380488 +0000 UTC m=+0.617959453 container attach e740819b5e9fda0fed13dd357d14dba68853e19836c638f0120d9a743bb18f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T17:21:17.381 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 17:21:17 smithi096 podman[106664]: 2024-04-25 17:21:17.017948756 +0000 UTC m=+0.666527723 container died e740819b5e9fda0fed13dd357d14dba68853e19836c638f0120d9a743bb18f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:21:17.717 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T17:21:17.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:17.756 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T17:21:17.757 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T17:21:17.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.1 2024-04-25T17:21:18.066 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:17 smithi097 systemd[1]: Stopping Ceph osd.1 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:18.484 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1[37175]: 2024-04-25T17:21:18.065+0000 7fcc599ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:18.485 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1[37175]: 2024-04-25T17:21:18.065+0000 7fcc599ab700 -1 osd.1 91 *** Got signal Terminated *** 2024-04-25T17:21:18.485 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1[37175]: 2024-04-25T17:21:18.065+0000 7fcc599ab700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:21:23.726 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:23 smithi097 podman[68757]: 2024-04-25 17:21:23.421852135 +0000 UTC m=+5.469980785 container died 5c82547aa2cdbe6a6e28e7b93b38c4322d4fe46d66197259e7ec5f2d290990f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:21:23.985 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:23 smithi097 podman[68757]: 2024-04-25 17:21:23.824763925 +0000 UTC m=+5.872892642 container cleanup 5c82547aa2cdbe6a6e28e7b93b38c4322d4fe46d66197259e7ec5f2d290990f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T17:21:23.985 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:23 smithi097 bash[68757]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1 2024-04-25T17:21:24.484 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:24 smithi097 podman[68774]: 2024-04-25 17:21:24.084776062 +0000 UTC m=+0.651966494 container remove 5c82547aa2cdbe6a6e28e7b93b38c4322d4fe46d66197259e7ec5f2d290990f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T17:21:24.838 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:24 smithi097 podman[68865]: 2024-04-25 17:21:24.403719967 +0000 UTC m=+0.018016022 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:21:24.838 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:24 smithi097 podman[68865]: 2024-04-25 17:21:24.519031205 +0000 UTC m=+0.133327245 container create de11223d544c3241f82123a9fbac051187a80cd90727b077215268a261cb76bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T17:21:24.838 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:24 smithi097 podman[68865]: 2024-04-25 17:21:24.738529948 +0000 UTC m=+0.352825996 container init de11223d544c3241f82123a9fbac051187a80cd90727b077215268a261cb76bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T17:21:24.839 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:24 smithi097 podman[68865]: 2024-04-25 17:21:24.741803918 +0000 UTC m=+0.356099963 container start de11223d544c3241f82123a9fbac051187a80cd90727b077215268a261cb76bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T17:21:24.839 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:24 smithi097 podman[68865]: 2024-04-25 17:21:24.838314345 +0000 UTC m=+0.452610388 container attach de11223d544c3241f82123a9fbac051187a80cd90727b077215268a261cb76bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T17:21:25.234 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 17:21:25 smithi097 podman[68865]: 2024-04-25 17:21:25.031759375 +0000 UTC m=+0.646055418 container died de11223d544c3241f82123a9fbac051187a80cd90727b077215268a261cb76bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:21:25.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T17:21:25.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:25.754 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T17:21:25.754 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T17:21:25.754 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.2 2024-04-25T17:21:26.031 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:25 smithi171 systemd[1]: Stopping Ceph osd.2 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:26.531 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2[38352]: 2024-04-25T17:21:26.072+0000 7f4332ae8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:26.531 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2[38352]: 2024-04-25T17:21:26.072+0000 7f4332ae8700 -1 osd.2 91 *** Got signal Terminated *** 2024-04-25T17:21:26.531 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2[38352]: 2024-04-25T17:21:26.072+0000 7f4332ae8700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:21:31.765 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:31 smithi171 podman[70564]: 2024-04-25 17:21:31.479886633 +0000 UTC m=+5.524943571 container died fe7254a019dd0c60c1afbf63a54a64b2bb84cbb2ce178dbc76d9d92ae2faa3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:21:32.031 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:31 smithi171 podman[70564]: 2024-04-25 17:21:31.864845909 +0000 UTC m=+5.909902846 container cleanup fe7254a019dd0c60c1afbf63a54a64b2bb84cbb2ce178dbc76d9d92ae2faa3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:32.031 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:31 smithi171 bash[70564]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2 2024-04-25T17:21:32.483 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:32 smithi171 podman[70581]: 2024-04-25 17:21:32.150322 +0000 UTC m=+0.665086096 container remove fe7254a019dd0c60c1afbf63a54a64b2bb84cbb2ce178dbc76d9d92ae2faa3b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T17:21:32.753 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:32 smithi171 podman[70677]: 2024-04-25 17:21:32.386194528 +0000 UTC m=+0.018937626 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:21:32.754 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:32 smithi171 podman[70677]: 2024-04-25 17:21:32.496784388 +0000 UTC m=+0.129527522 container create 41e2db88cb9573250ccc0984e0d63d3128e5749967703f8e2088fd148289128d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:21:33.031 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:32 smithi171 podman[70677]: 2024-04-25 17:21:32.75413424 +0000 UTC m=+0.386877330 container init 41e2db88cb9573250ccc0984e0d63d3128e5749967703f8e2088fd148289128d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:33.031 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:32 smithi171 podman[70677]: 2024-04-25 17:21:32.75739833 +0000 UTC m=+0.390141423 container start 41e2db88cb9573250ccc0984e0d63d3128e5749967703f8e2088fd148289128d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:21:33.032 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:32 smithi171 podman[70677]: 2024-04-25 17:21:32.854014947 +0000 UTC m=+0.486758041 container attach 41e2db88cb9573250ccc0984e0d63d3128e5749967703f8e2088fd148289128d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T17:21:33.325 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 25 17:21:33 smithi171 podman[70703]: 2024-04-25 17:21:33.078847281 +0000 UTC m=+0.020551872 container died 41e2db88cb9573250ccc0984e0d63d3128e5749967703f8e2088fd148289128d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T17:21:33.674 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T17:21:33.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:33.712 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T17:21:33.712 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T17:21:33.712 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.3 2024-04-25T17:21:34.189 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:33 smithi096 systemd[1]: Stopping Ceph osd.3 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:34.462 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:21:34.208+0000 7f92bb04b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:34.463 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:21:34.208+0000 7f92bb04b700 -1 osd.3 91 *** Got signal Terminated *** 2024-04-25T17:21:34.463 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:34 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:21:34.208+0000 7f92bb04b700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:21:35.689 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:35 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:21:35.620+0000 7f92b7601700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.760151+0000 front 2024-04-25T17:21:09.760123+0000 (oldest deadline 2024-04-25T17:21:35.560857+0000) 2024-04-25T17:21:36.781 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:36.285+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:36.939 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:36 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:21:36.599+0000 7f92b7601700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.760151+0000 front 2024-04-25T17:21:09.760123+0000 (oldest deadline 2024-04-25T17:21:35.560857+0000) 2024-04-25T17:21:37.700 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:37.314+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:37.939 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:37 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:21:37.621+0000 7f92b7601700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.760151+0000 front 2024-04-25T17:21:09.760123+0000 (oldest deadline 2024-04-25T17:21:35.560857+0000) 2024-04-25T17:21:38.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:37.699+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:38.673 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:38.318+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:38.939 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:38 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3[50455]: 2024-04-25T17:21:38.668+0000 7f92b7601700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.760151+0000 front 2024-04-25T17:21:09.760123+0000 (oldest deadline 2024-04-25T17:21:35.560857+0000) 2024-04-25T17:21:38.940 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:38 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:38.643+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:39.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:38.673+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:39.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:39 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:39.127+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:39.720 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:39.323+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:39.925 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:39 smithi096 podman[106772]: 2024-04-25 17:21:39.621449258 +0000 UTC m=+5.715816729 container died 46ddbe251f8ad28b3d39ac129d5cdd89f86055cda345d539de9539fbb7124dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T17:21:39.925 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:39 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:39.649+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:40.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:39.719+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:40.189 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:40.153+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:40.190 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 podman[106772]: 2024-04-25 17:21:40.024175185 +0000 UTC m=+6.118542658 container cleanup 46ddbe251f8ad28b3d39ac129d5cdd89f86055cda345d539de9539fbb7124dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:21:40.190 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 bash[106772]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3 2024-04-25T17:21:40.587 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 podman[106802]: 2024-04-25 17:21:40.267127155 +0000 UTC m=+0.647716252 container remove 46ddbe251f8ad28b3d39ac129d5cdd89f86055cda345d539de9539fbb7124dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T17:21:40.700 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:40.324+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:40.854 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:40 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:40.611+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:40.854 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 podman[106890]: 2024-04-25 17:21:40.490265291 +0000 UTC m=+0.017862837 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:21:40.854 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 podman[106890]: 2024-04-25 17:21:40.626556775 +0000 UTC m=+0.154154352 container create 01caffd45d7a61d15a8367df6d964855b37c47ba7af3892dde442eb4ba5f6c67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:21:41.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:40.699+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:41.154 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 podman[106890]: 2024-04-25 17:21:40.854447271 +0000 UTC m=+0.382044806 container init 01caffd45d7a61d15a8367df6d964855b37c47ba7af3892dde442eb4ba5f6c67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T17:21:41.154 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 podman[106890]: 2024-04-25 17:21:40.857807882 +0000 UTC m=+0.385405420 container start 01caffd45d7a61d15a8367df6d964855b37c47ba7af3892dde442eb4ba5f6c67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T17:21:41.154 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:40 smithi096 podman[106890]: 2024-04-25 17:21:40.97135119 +0000 UTC m=+0.498948731 container attach 01caffd45d7a61d15a8367df6d964855b37c47ba7af3892dde442eb4ba5f6c67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-25T17:21:41.439 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 17:21:41 smithi096 podman[106890]: 2024-04-25 17:21:41.153853853 +0000 UTC m=+0.681451390 container died 01caffd45d7a61d15a8367df6d964855b37c47ba7af3892dde442eb4ba5f6c67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:21:41.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:41 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:41.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:41.674 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:41.328+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:41.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:41 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:41.442+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:41.841 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:41 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:41.617+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:41.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T17:21:41.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:41.905 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T17:21:41.905 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T17:21:41.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.4 2024-04-25T17:21:42.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:41.674+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:42.212 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:41 smithi097 systemd[1]: Stopping Ceph osd.4 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:42.482 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:42 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4[43840]: 2024-04-25T17:21:42.211+0000 7f49ab8f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:42.482 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:42 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4[43840]: 2024-04-25T17:21:42.211+0000 7f49ab8f4700 -1 osd.4 91 *** Got signal Terminated *** 2024-04-25T17:21:42.482 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:42 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4[43840]: 2024-04-25T17:21:42.211+0000 7f49ab8f4700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:21:42.642 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:42 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:42.194+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:42.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:42.357+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:42.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:42 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:42.482+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:42.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:42 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:42.641+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:43.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:42.678+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:43.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:43 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:43.162+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:43.670 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:43 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:43.335+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:43.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:43 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:43.475+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:43.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:43 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:43.612+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:44.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:43 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:43.670+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:44.596 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:44 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:44.198+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:44.646 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:44 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:44.347+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:44.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:44 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:44.467+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:44.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:44 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:44.595+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:45.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:44 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:44.645+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:45.631 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:45 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:45.203+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:45.687 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:45 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:45.371+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:45.734 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:45 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:45.423+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:45.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:45 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:45.653+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:45.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:45 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:45.630+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:45.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:45 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:45.630+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:46.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:45 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:45.687+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:46.650 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:46 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:46.236+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:46.719 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:46 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:46.329+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:46.734 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:46 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4[43840]: 2024-04-25T17:21:46.615+0000 7f49a3ea2700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.241286+0000 front 2024-04-25T17:21:22.241258+0000 (oldest deadline 2024-04-25T17:21:45.741438+0000) 2024-04-25T17:21:46.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:46 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:46.621+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:46.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:46 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:46.400+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:46.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:46 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:46.400+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:46.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:46 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:46.649+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:46.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:46 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:46.649+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:47.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:46 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:46.719+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:47.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:46 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:46.719+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:47.627 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:47 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:47.257+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:47.735 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:47 smithi097 podman[68988]: 2024-04-25 17:21:47.682097991 +0000 UTC m=+5.587424570 container died 55bd02cebc10d703b0abde6d557c618c08e3439cf074c2adef5b9ec7121dad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:21:47.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:47 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:47.591+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:47.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:47 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:47.437+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:47.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:47 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:47.437+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:47.762 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:47 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:47.306+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:47.762 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:47 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:47.306+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:47.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:47 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:47.626+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:47.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:47 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:47.626+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:48.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:47 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:47.849+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:48.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:47 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:47.762+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:48.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:47 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:47.762+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:48.347 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:48 smithi097 podman[68988]: 2024-04-25 17:21:48.079431358 +0000 UTC m=+5.984757928 container cleanup 55bd02cebc10d703b0abde6d557c618c08e3439cf074c2adef5b9ec7121dad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T17:21:48.347 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:48 smithi097 bash[68988]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4 2024-04-25T17:21:48.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:48 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:48.273+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:48.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:48 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:48.273+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:48.634 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:48 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:48.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:48.634 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:48 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:48.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:48.658 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:48 smithi097 podman[69005]: 2024-04-25 17:21:48.347421908 +0000 UTC m=+0.667688793 container remove 55bd02cebc10d703b0abde6d557c618c08e3439cf074c2adef5b9ec7121dad71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T17:21:48.659 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:48 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:48.550+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:48.659 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:48 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:48.416+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:48.659 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:48 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:48.416+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:48.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:48 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:48.633+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:48.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:48 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:48.633+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:48.942 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:48 smithi097 podman[69099]: 2024-04-25 17:21:48.561414396 +0000 UTC m=+0.017961476 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:21:48.942 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:48 smithi097 podman[69099]: 2024-04-25 17:21:48.690291608 +0000 UTC m=+0.146838668 container create 340245b5a331321e251fcbf2bf6aba18ea97faa2d807059eec03e60c433d0bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T17:21:48.943 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:48 smithi097 podman[69099]: 2024-04-25 17:21:48.942904408 +0000 UTC m=+0.399451467 container init 340245b5a331321e251fcbf2bf6aba18ea97faa2d807059eec03e60c433d0bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T17:21:49.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:48 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:48.899+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:49.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:48 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:48.756+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:49.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:48 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:48.756+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:49.234 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:48 smithi097 podman[69099]: 2024-04-25 17:21:48.946006697 +0000 UTC m=+0.402553768 container start 340245b5a331321e251fcbf2bf6aba18ea97faa2d807059eec03e60c433d0bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T17:21:49.235 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:49 smithi097 podman[69099]: 2024-04-25 17:21:49.042914094 +0000 UTC m=+0.499461161 container attach 340245b5a331321e251fcbf2bf6aba18ea97faa2d807059eec03e60c433d0bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T17:21:49.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:49 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:49.256+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:49.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:49 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:49.256+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:49.554 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 17:21:49 smithi097 podman[69099]: 2024-04-25 17:21:49.246186126 +0000 UTC m=+0.702733208 container died 340245b5a331321e251fcbf2bf6aba18ea97faa2d807059eec03e60c433d0bc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T17:21:49.554 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:49 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:49.402+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:49.554 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:49 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:49.402+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:49.642 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:49 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:49.313+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:49.642 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:49 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:49.313+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:49.880 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:49 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:49.584+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:49.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T17:21:49.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:49.939 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T17:21:49.939 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T17:21:49.939 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.5 2024-04-25T17:21:49.940 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:49 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:49.641+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:49.940 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:49 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:49.641+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:50.031 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:49 smithi171 systemd[1]: Stopping Ceph osd.5 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:50.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:49 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:49.911+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:50.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:49 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:49.755+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:50.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:49 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:49.755+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:50.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:50.243+0000 7f659b031700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:50.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:50.243+0000 7f659b031700 -1 osd.5 91 *** Got signal Terminated *** 2024-04-25T17:21:50.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:50.243+0000 7f659b031700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:21:50.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:50.282+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:50.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:50.282+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:50.645 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:50.285+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:50.646 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:50.285+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:50.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:50 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:50.587+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:50.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:50 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:50.433+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:50.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:50 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:50.433+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:50.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:50.645+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:50.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:50.645+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:51.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:50.862+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:51.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:50.766+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:51.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:50 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:50.766+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:51.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:51 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:51.234+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:51.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:51 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:51.234+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:51.620 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:51 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:51.287+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:51.621 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:51 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:51.287+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:51.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:51 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:51.569+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:51.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:51 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:51.394+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:51.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:51 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:51.395+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:51.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:51 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:51.620+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:51.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:51 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:51.620+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:52.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:51 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:51.849+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:52.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:51 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:51.738+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:52.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:51 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:51.738+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:52.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:52.243+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:52.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:52.243+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:52.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:52.243+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.961267+0000 front 2024-04-25T17:21:30.961300+0000 (oldest deadline 2024-04-25T17:21:52.061396+0000) 2024-04-25T17:21:52.645 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:52 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:52.284+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:52.645 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:52 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:52.284+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:52.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:52 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:52.585+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:52.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:52 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:52.585+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:52.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:52 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:52.422+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:52.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:52 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:52.422+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:52.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:52 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:52.645+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:52.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:52 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:52.645+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:53.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:52.885+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:53.242 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:52.885+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:21:53.242 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:52.787+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:53.242 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:52.787+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:53.242 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:52 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:52.787+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:21:53.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:53.241+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:53.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:53.241+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:53.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:53.241+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.961267+0000 front 2024-04-25T17:21:30.961300+0000 (oldest deadline 2024-04-25T17:21:52.061396+0000) 2024-04-25T17:21:53.671 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:53 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:53.247+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:53.671 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:53 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:53.247+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:53.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:53 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:53.578+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:53.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:53 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:53.578+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:53.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:53 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:53.450+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:53.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:53 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:53.450+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:53.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:53 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:53.670+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:53.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:53 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:53.670+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:53.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:53 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:53.670+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:21:54.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:53.934+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:54.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:53.934+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:21:54.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:53.738+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:54.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:53.738+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:54.032 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:53 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:53.738+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:21:54.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:54.266+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.559087+0000 front 2024-04-25T17:21:11.559057+0000 (oldest deadline 2024-04-25T17:21:35.659057+0000) 2024-04-25T17:21:54.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:54.266+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.960158+0000 front 2024-04-25T17:21:20.960389+0000 (oldest deadline 2024-04-25T17:21:46.860186+0000) 2024-04-25T17:21:54.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5[45305]: 2024-04-25T17:21:54.266+0000 7f65935df700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.961267+0000 front 2024-04-25T17:21:30.961300+0000 (oldest deadline 2024-04-25T17:21:52.061396+0000) 2024-04-25T17:21:54.634 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:54 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:54.253+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:54.634 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:54 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:54.253+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:54.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:54 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:54.535+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:54.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:54 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:54.535+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:54.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:54 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:54.427+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:54.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:54 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:54.427+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:54.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:54 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:54.633+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:54.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:54 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:54.633+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:54.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:54 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:54.633+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:21:55.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:54.960+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:55.031 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:54.960+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:21:55.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:54.716+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:55.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:54.716+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:55.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:54 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:54.716+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:21:55.610 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:55.227+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:55.611 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:55.227+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:55.611 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:55.227+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:21:55.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:55 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:55.496+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:55.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:55 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:55.496+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:55.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:55 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:55.454+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:55.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:55 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:55.454+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:55.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:55 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:55.454+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:21:55.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:55.610+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:55.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:55.610+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:55.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:55.610+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:21:55.964 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:55 smithi171 podman[70793]: 2024-04-25 17:21:55.758930378 +0000 UTC m=+5.616250938 container died bfd33f2472eab72944b73d87878c1ec43108de8e9c519e875a81d8320d205fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T17:21:55.964 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:55 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:55.701+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:55.964 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:55 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:55.701+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:55.964 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:55 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:55.701+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:21:56.281 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:56 smithi171 podman[70793]: 2024-04-25 17:21:56.169660209 +0000 UTC m=+6.026980767 container cleanup bfd33f2472eab72944b73d87878c1ec43108de8e9c519e875a81d8320d205fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T17:21:56.281 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:56 smithi171 bash[70793]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5 2024-04-25T17:21:56.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:55 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:55.963+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:56.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:55 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:55.963+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:21:56.590 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:56.192+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:56.590 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:56.192+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:56.590 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:56.192+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:21:56.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:56 smithi171 podman[70812]: 2024-04-25 17:21:56.395843966 +0000 UTC m=+0.632081926 container remove bfd33f2472eab72944b73d87878c1ec43108de8e9c519e875a81d8320d205fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T17:21:56.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:56 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:56.453+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:56.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:56 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:56.453+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:56.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:56 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:56.496+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:56.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:56 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:56.496+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:56.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:56 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:56.496+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:21:56.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:56.590+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:56.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:56.590+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:56.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:56 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:56.590+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:21:56.954 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:56 smithi171 podman[70897]: 2024-04-25 17:21:56.698874382 +0000 UTC m=+0.017852350 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:21:56.954 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:56 smithi171 podman[70897]: 2024-04-25 17:21:56.821587795 +0000 UTC m=+0.140565745 container create f557ad2b41e0c75648412119b79067465abef48853e0edd7c24b98e33d7cc5c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:56.954 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:56 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:56.678+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:56.954 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:56 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:56.678+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:56.954 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:56 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:56.678+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:21:57.224 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:57 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:57.002+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:57.224 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:57 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:57.002+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:21:57.225 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:57 smithi171 podman[70897]: 2024-04-25 17:21:57.13259983 +0000 UTC m=+0.451577781 container init f557ad2b41e0c75648412119b79067465abef48853e0edd7c24b98e33d7cc5c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T17:21:57.225 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:57 smithi171 podman[70897]: 2024-04-25 17:21:57.13572043 +0000 UTC m=+0.454698381 container start f557ad2b41e0c75648412119b79067465abef48853e0edd7c24b98e33d7cc5c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:57.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:57 smithi171 podman[70897]: 2024-04-25 17:21:57.22435638 +0000 UTC m=+0.543334343 container attach f557ad2b41e0c75648412119b79067465abef48853e0edd7c24b98e33d7cc5c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T17:21:57.531 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 25 17:21:57 smithi171 podman[70897]: 2024-04-25 17:21:57.429921362 +0000 UTC m=+0.748899313 container died f557ad2b41e0c75648412119b79067465abef48853e0edd7c24b98e33d7cc5c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:21:57.595 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:57 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:57.223+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:57.596 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:57 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:57.223+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:57.596 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:57 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:57.223+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:21:57.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:57 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:57.434+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:57.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:57 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:57.434+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:57.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:57 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:57.487+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:57.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:57 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:57.487+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:57.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:57 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:57.487+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:21:57.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:57 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:57.595+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:57.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:57 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:57.595+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:57.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:57 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:57.595+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:21:58.030 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:57 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:57.699+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:58.030 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:57 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:57.699+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:58.031 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:57 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:57.699+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:21:58.053 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T17:21:58.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:21:58.092 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T17:21:58.092 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T17:21:58.093 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.6 2024-04-25T17:21:58.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:58 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:58.033+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:58.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:58 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:58.033+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:21:58.410 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:58 smithi096 systemd[1]: Stopping Ceph osd.6 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:21:58.410 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:58.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:58.410 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:58.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:58.410 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:58.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:21:58.689 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:58.409+0000 7f3b77c1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:21:58.690 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:58.409+0000 7f3b77c1c700 -1 osd.6 91 *** Got signal Terminated *** 2024-04-25T17:21:58.690 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:58.409+0000 7f3b77c1c700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:21:58.690 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:58.595+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:58.690 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:58.595+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:58.690 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:58 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:58.595+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:21:58.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:58.443+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:58.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:58.443+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:58.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:58.504+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:58.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:58.504+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:58.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:58 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:58.504+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:21:59.001 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:58 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:58.743+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:21:59.001 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:58 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:58.743+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:21:59.001 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:58 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:58.743+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:21:59.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:59.000+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:21:59.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:59.000+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:21:59.577 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:59.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:21:59.577 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:59.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:21:59.577 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:21:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:21:59.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:21:59.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:59.577+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:21:59.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:59.577+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:21:59.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:21:59 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:21:59.577+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:21:59.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:59 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:59.491+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:21:59.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:21:59 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:21:59.491+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:21:59.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:59 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:59.512+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:21:59.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:59 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:59.512+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:21:59.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:21:59 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:21:59.512+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:00.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:59.790+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:00.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:59.790+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:00.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:21:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:21:59.790+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:00.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:59.975+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:00.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:21:59 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:21:59.975+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:00.605 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:00.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:00.605 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:00.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:00.605 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:00.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:00.605 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:00.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:00.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:00.604+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:22:00.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:00.604+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:22:00.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:00.604+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:22:00.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:00.604+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.672514+0000 front 2024-04-25T17:21:35.672546+0000 (oldest deadline 2024-04-25T17:21:59.772694+0000) 2024-04-25T17:22:00.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:00 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:00.522+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:00.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:00 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:00.522+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:00.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:00 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:00.525+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:00.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:00 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:00.525+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:00.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:00 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:00.525+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:01.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:00 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:00.968+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:01.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:00 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:00.968+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:01.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:00 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:00.796+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:01.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:00 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:00.796+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:01.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:00 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:00.796+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:01.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:00 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:00.796+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:01.608 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:01.200+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:01.608 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:01.200+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:01.608 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:01.200+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:01.608 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:01.200+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:01.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:01.607+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:22:01.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:01.607+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:22:01.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:01.607+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:22:01.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:01 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:01.607+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.672514+0000 front 2024-04-25T17:21:35.672546+0000 (oldest deadline 2024-04-25T17:21:59.772694+0000) 2024-04-25T17:22:01.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:01 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:01.524+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:01.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:01 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:01.524+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:01.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:01 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:01.502+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:01.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:01 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:01.502+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:01.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:01 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:01.502+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:02.182 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:01 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:01.921+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:02.183 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:01 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:01.921+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:02.183 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:01 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:01.921+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:02.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:01 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:01.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:02.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:01 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:01.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:02.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:01 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:01.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:02.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:01 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:01.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:02.652 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:02.204+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:02.652 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:02.204+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:02.652 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:02.204+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:02.652 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:02.204+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:02.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:02.557+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:02.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:02.557+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:02.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:02.483+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:02.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:02.483+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:02.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:02.483+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:02.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:02 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:02.483+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:02.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:02.651+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:13.067726+0000 front 2024-04-25T17:21:13.067695+0000 (oldest deadline 2024-04-25T17:21:38.367930+0000) 2024-04-25T17:22:02.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:02.651+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.469390+0000 front 2024-04-25T17:21:22.469191+0000 (oldest deadline 2024-04-25T17:21:44.769162+0000) 2024-04-25T17:22:02.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:02.651+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.570927+0000 front 2024-04-25T17:21:30.570998+0000 (oldest deadline 2024-04-25T17:21:53.470942+0000) 2024-04-25T17:22:02.939 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:02 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6[57484]: 2024-04-25T17:22:02.651+0000 7f3b741d2700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.672514+0000 front 2024-04-25T17:21:35.672546+0000 (oldest deadline 2024-04-25T17:21:59.772694+0000) 2024-04-25T17:22:03.196 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:02 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:02.914+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:03.196 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:02 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:02.914+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:03.196 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:02 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:02.914+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:02 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:02.807+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:02 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:02.807+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:02 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:02.807+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:03.196 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:02 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:02.807+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:03.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:03 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:03.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:03.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:03 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:03.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:03.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:03 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:03.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:03.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:03 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:03.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:03.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:03.549+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:03.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:03.549+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:03.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:03.478+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:03.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:03.478+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:03.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:03.478+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:03.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:03 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:03.478+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:04.160 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:03 smithi096 podman[106995]: 2024-04-25 17:22:03.879848295 +0000 UTC m=+5.598219709 container died f131721884f14760b6a5387e696e69e7f4462e12e84b83bb20256b6c60f18b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:22:04.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:03 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:03.925+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:04.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:03 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:03.925+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:04.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:03 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:03.925+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:04.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:03 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:03.813+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:04.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:03 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:03.813+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:04.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:03 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:03.813+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:04.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:03 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:03.813+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:04.439 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:04 smithi096 podman[106995]: 2024-04-25 17:22:04.259322203 +0000 UTC m=+5.977693611 container cleanup f131721884f14760b6a5387e696e69e7f4462e12e84b83bb20256b6c60f18b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:22:04.439 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:04 smithi096 bash[106995]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6 2024-04-25T17:22:04.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:04 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:04.205+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:04.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:04 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:04.205+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:04.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:04 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:04.205+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:04.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:04 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:04.205+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:04.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:04.541+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:04.735 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:04.541+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:04.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:04.455+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:04.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:04.455+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:04.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:04.455+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:04.735 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:04.455+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:04.827 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:04 smithi096 podman[107012]: 2024-04-25 17:22:04.510651927 +0000 UTC m=+0.624625839 container remove f131721884f14760b6a5387e696e69e7f4462e12e84b83bb20256b6c60f18b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-25T17:22:05.106 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:04 smithi096 podman[107103]: 2024-04-25 17:22:04.730888993 +0000 UTC m=+0.018044549 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:22:05.106 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:04 smithi096 podman[107103]: 2024-04-25 17:22:04.836409151 +0000 UTC m=+0.123564697 container create bc1c7d597a4cafaf26f4ccc34a7b3b434f9c91ae4e423e1e3104cc6c7294065e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:22:05.106 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:05 smithi096 podman[107103]: 2024-04-25 17:22:05.106106253 +0000 UTC m=+0.393261804 container init bc1c7d597a4cafaf26f4ccc34a7b3b434f9c91ae4e423e1e3104cc6c7294065e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:22:05.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:04 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:04.972+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:05.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:04 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:04.972+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:05.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:04 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:04.972+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:05.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:04 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:04.824+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:05.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:04 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:04.824+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:05.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:04 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:04.824+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:05.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:04 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:04.824+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:05.401 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:05 smithi096 podman[107103]: 2024-04-25 17:22:05.10929119 +0000 UTC m=+0.396446746 container start bc1c7d597a4cafaf26f4ccc34a7b3b434f9c91ae4e423e1e3104cc6c7294065e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T17:22:05.401 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:05 smithi096 podman[107103]: 2024-04-25 17:22:05.197837956 +0000 UTC m=+0.484993513 container attach bc1c7d597a4cafaf26f4ccc34a7b3b434f9c91ae4e423e1e3104cc6c7294065e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T17:22:05.401 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:05 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:05.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:05.401 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:05 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:05.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:05.401 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:05 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:05.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:05.401 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:05 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:05.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:05.689 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:05 smithi096 podman[107103]: 2024-04-25 17:22:05.401373726 +0000 UTC m=+0.688529273 container died bc1c7d597a4cafaf26f4ccc34a7b3b434f9c91ae4e423e1e3104cc6c7294065e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T17:22:05.956 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 17:22:05 smithi096 podman[107129]: 2024-04-25 17:22:05.934444177 +0000 UTC m=+0.527707169 container remove bc1c7d597a4cafaf26f4ccc34a7b3b434f9c91ae4e423e1e3104cc6c7294065e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T17:22:05.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T17:22:05.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:05.589+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:05.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:05.589+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:05.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:05.589+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.092790+0000 front 2024-04-25T17:21:37.092911+0000 (oldest deadline 2024-04-25T17:22:05.293124+0000) 2024-04-25T17:22:05.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:05.495+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:05.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:05.495+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:05.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:05.495+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:05.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:05 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:05.495+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:05.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:05.998 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T17:22:05.998 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T17:22:05.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.7 2024-04-25T17:22:06.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:05.968+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:06.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:05.968+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:06.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:05.968+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:06.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:05.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:06.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:05.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:06.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:05.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:06.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:05 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:05.828+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:06.298 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:06 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:06.137+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:06.298 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:06 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:06.137+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:06.298 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:06 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:06.137+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:06.298 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:06 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:06.137+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:06.309 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:06 smithi097 systemd[1]: Stopping Ceph osd.7 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:06.570 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:06.308+0000 7f7253261700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:06.570 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:06.308+0000 7f7253261700 -1 osd.7 91 *** Got signal Terminated *** 2024-04-25T17:22:06.570 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:06.308+0000 7f7253261700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:22:06.570 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:06.523+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:06.570 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:06.523+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:06.570 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:06.523+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:06.570 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:06.523+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:06.984 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:06.602+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:06.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:06.602+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:06.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:06 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:06.602+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.092790+0000 front 2024-04-25T17:21:37.092911+0000 (oldest deadline 2024-04-25T17:22:05.293124+0000) 2024-04-25T17:22:07.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:07.005+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:07.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:07.005+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:07.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:07.005+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:07.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:06 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:06.851+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:07.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:06 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:06.851+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:07.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:06 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:06.851+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:07.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:06 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:06.851+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:07.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:07.153+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:07.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:07.153+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:07.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:07.153+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:07.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:07 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:07.153+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:07.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:07.617+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:07.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:07.617+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:07.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:07.617+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.092790+0000 front 2024-04-25T17:21:37.092911+0000 (oldest deadline 2024-04-25T17:22:05.293124+0000) 2024-04-25T17:22:07.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:07.508+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:07.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:07.508+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:07.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:07.508+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:07.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:07 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:07.508+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:08.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:07.983+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:08.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:07.983+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:08.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:07.983+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:08.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:07.983+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:08.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:07.861+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:08.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:07.861+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:08.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:07.861+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:08.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:07 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:07.861+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:08.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:08 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:08.176+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:08.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:08 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:08.176+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:08.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:08 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:08.176+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:08.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:08 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:08.176+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:08.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:08.658+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:08.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:08.658+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:08.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:08.658+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.092790+0000 front 2024-04-25T17:21:37.092911+0000 (oldest deadline 2024-04-25T17:22:05.293124+0000) 2024-04-25T17:22:08.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:08.494+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:08.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:08.494+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:08.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:08.494+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:08.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:08.494+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:09.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:08.855+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:09.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:08.855+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:09.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:08.855+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:09.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:08.855+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:09.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:08.955+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:09.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:08.955+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:09.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:08.955+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:09.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:08 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:08.955+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:09.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:09.192+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:09.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:09.192+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:09.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:09.192+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:09.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:09 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:09.192+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:09.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:09.706+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:09.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:09.706+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:09.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:09.706+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.092790+0000 front 2024-04-25T17:21:37.092911+0000 (oldest deadline 2024-04-25T17:22:05.293124+0000) 2024-04-25T17:22:09.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:09.501+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:09.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:09.501+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:09.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:09.501+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:09.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:09 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:09.501+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:10.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:09.965+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:10.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:09.965+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:10.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:09.965+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:10.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:09.965+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:10.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:09.900+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:10.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:09.900+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:10.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:09.900+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:10.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:09 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:09.900+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:10.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:10.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:10.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:10.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:10.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:10.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:10.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:10.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:10.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:10 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:10.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:10.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:10.704+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.790025+0000 front 2024-04-25T17:21:20.790058+0000 (oldest deadline 2024-04-25T17:21:44.790744+0000) 2024-04-25T17:22:10.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:10.704+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.491735+0000 front 2024-04-25T17:21:29.491889+0000 (oldest deadline 2024-04-25T17:21:52.391677+0000) 2024-04-25T17:22:10.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7[50544]: 2024-04-25T17:22:10.704+0000 7f724b80f700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.092790+0000 front 2024-04-25T17:21:37.092911+0000 (oldest deadline 2024-04-25T17:22:05.293124+0000) 2024-04-25T17:22:10.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:10.544+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:10.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:10.544+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:10.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:10.544+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:10.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:10.544+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:11.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:10.964+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:11.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:10.964+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:11.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:10.964+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:11.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:10.964+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:11.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:10.924+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:11.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:10.924+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:11.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:10.924+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:11.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:10.924+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:11.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:10 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:10.924+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:11.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:11 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:11.230+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:11.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:11 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:11.230+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:11.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:11 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:11.230+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:11.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:11 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:11.230+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:11.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:11 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:11.230+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:11.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:11 smithi097 podman[69205]: 2024-04-25 17:22:11.725093115 +0000 UTC m=+5.536062979 container died a5f7d62f56a0765abc153abd2e17b060ae9f294bab8f8e596ded600c7cc71339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T17:22:11.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:11.506+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:11.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:11.506+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:11.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:11.506+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:11.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:11.506+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:11.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:11 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:11.506+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:12.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:11.945+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:12.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:11.945+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:12.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:11.945+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:12.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:11.945+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:12.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:11.937+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:12.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:11.937+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:12.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:11.937+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:12.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:11.937+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:12.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:11 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:11.937+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:12.352 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:12 smithi097 podman[69205]: 2024-04-25 17:22:12.101487248 +0000 UTC m=+5.912457162 container cleanup a5f7d62f56a0765abc153abd2e17b060ae9f294bab8f8e596ded600c7cc71339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:22:12.352 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:12 smithi097 bash[69205]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7 2024-04-25T17:22:12.666 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:12 smithi097 podman[69234]: 2024-04-25 17:22:12.352721471 +0000 UTC m=+0.630777023 container remove a5f7d62f56a0765abc153abd2e17b060ae9f294bab8f8e596ded600c7cc71339 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T17:22:12.667 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:12.538+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:12.667 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:12.538+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:12.667 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:12.538+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:12.667 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:12.538+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:12.667 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:12 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:12.538+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:12.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:12.191+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:12.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:12.191+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:12.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:12.191+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:12.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:12.191+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:12.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:12 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:12.191+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:12.973 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:12 smithi097 podman[69323]: 2024-04-25 17:22:12.569715936 +0000 UTC m=+0.017644804 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:22:12.973 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:12 smithi097 podman[69323]: 2024-04-25 17:22:12.711927776 +0000 UTC m=+0.159856633 container create 4c354395193b8d8be019dba01c77d00473f7b56151bf6af358d7369d9005aa5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T17:22:12.973 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:12 smithi097 podman[69323]: 2024-04-25 17:22:12.973415271 +0000 UTC m=+0.421344126 container init 4c354395193b8d8be019dba01c77d00473f7b56151bf6af358d7369d9005aa5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T17:22:13.235 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:12 smithi097 podman[69323]: 2024-04-25 17:22:12.977010017 +0000 UTC m=+0.424938878 container start 4c354395193b8d8be019dba01c77d00473f7b56151bf6af358d7369d9005aa5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T17:22:13.235 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:13 smithi097 podman[69323]: 2024-04-25 17:22:13.065188441 +0000 UTC m=+0.513117312 container attach 4c354395193b8d8be019dba01c77d00473f7b56151bf6af358d7369d9005aa5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T17:22:13.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:12.938+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:13.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:12.938+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:13.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:12.938+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:13.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:12.938+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:13.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:12.933+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:13.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:12.933+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:13.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:12.933+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:13.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:12.933+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:13.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:12 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:12.933+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:13.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:13.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:13.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:13.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:13.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:13.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:13.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:13.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:13.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:13 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:13.177+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:13.560 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 17:22:13 smithi097 podman[69323]: 2024-04-25 17:22:13.270080914 +0000 UTC m=+0.718009775 container died 4c354395193b8d8be019dba01c77d00473f7b56151bf6af358d7369d9005aa5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T17:22:13.560 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:13.516+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:13.560 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:13.516+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:13.560 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:13.516+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:13.560 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:13.516+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:13.560 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:13 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:13.516+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:13.891 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T17:22:13.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:13.931 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T17:22:13.932 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T17:22:13.932 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.8 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:13.893+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:13.893+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:13.893+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:13.893+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:13.893+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:13 smithi171 systemd[1]: Stopping Ceph osd.8 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:13.982+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:13.982+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:14.252 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:13.982+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:14.253 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:13 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:13.982+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:14.253 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:14.251+0000 7f0fedf0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:14.253 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:14.251+0000 7f0fedf0b700 -1 osd.8 91 *** Got signal Terminated *** 2024-04-25T17:22:14.253 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:14.251+0000 7f0fedf0b700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:22:14.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:14.175+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:14.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:14.175+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:14.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:14.175+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:14.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:14.175+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:14.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:14 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:14.175+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:14.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:14 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:14.562+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:14.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:14 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:14.562+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:14.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:14 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:14.562+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:14.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:14 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:14.562+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:14.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:14 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:14.562+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:15.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:14.994+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:15.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:14.994+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:15.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:14.994+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:15.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:14.994+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:15.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:14.927+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:15.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:14.927+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:15.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:14.927+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:15.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:14.927+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:15.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:14 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:14.927+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:15.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:15.188+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:15.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:15.188+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:15.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:15.188+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:15.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:15.188+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:15.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:15 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:15.188+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:15.984 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:15.590+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:15.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:15.590+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:15.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:15.590+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:15.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:15.590+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:15.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:15.590+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:16.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:15.979+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:16.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:15.979+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:16.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:15.979+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:16.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:15.979+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:16.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:15.954+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:16.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:15.954+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:16.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:15.954+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:16.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:15.954+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:16.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:15 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:15.954+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:16.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:16 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:16.226+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:16.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:16 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:16.226+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:16.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:16 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:16.226+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:16.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:16 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:16.226+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:16.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:16 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:16.226+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:16.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:16.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:16.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:16.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:16.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:16.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:16.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:16.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:16.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:16 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:16.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:17.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:16.966+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:17.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:16.966+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:17.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:16.966+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:17.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:16.966+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:17.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:16.939+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:17.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:16.939+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:17.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:16.939+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:17.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:16.939+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:17.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:16 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:16.939+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:17.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:17.260+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:17.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:17.260+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:17.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:17.260+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:17.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:17.260+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:17.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:17 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:17.260+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:17.984 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:17 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:17.663+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:17.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:17 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:17.663+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:17.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:17 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:17.663+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:17.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:17 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:17.663+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:17.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:17 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:17.663+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:18.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:17.992+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:18.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:17.992+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:18.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:17.992+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:18.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:17.992+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:18.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:17.975+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:18.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:17.975+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:18.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:17.975+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:18.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:17.975+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:18.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:17.975+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:18.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:17 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:17.975+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:18.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:18.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:18.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:18.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:18.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:18.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:18.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:18.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:18.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:18 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:18.211+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:18.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:18.650+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:18.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:18.650+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:18.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:18.650+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:18.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:18.650+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:18.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:18.650+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:18.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:18 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:18.650+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:19.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:18.970+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.819411+0000 front 2024-04-25T17:21:22.819143+0000 (oldest deadline 2024-04-25T17:21:46.919166+0000) 2024-04-25T17:22:19.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:18.970+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.820024+0000 front 2024-04-25T17:21:29.820237+0000 (oldest deadline 2024-04-25T17:21:52.720160+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:18.970+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:38.021263+0000 front 2024-04-25T17:21:38.021166+0000 (oldest deadline 2024-04-25T17:22:01.520975+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8[52251]: 2024-04-25T17:22:18.970+0000 7f0fe64b9700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:43.222195+0000 front 2024-04-25T17:21:43.222120+0000 (oldest deadline 2024-04-25T17:22:07.921995+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:18.964+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:18.964+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:18.964+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:18.964+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:18.964+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:19.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:18 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:18.964+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:19.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:19.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:19.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:19.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:19.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:19.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:19.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:19.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:19.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:19 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:19.201+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:19.978 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:19 smithi171 podman[71015]: 2024-04-25 17:22:19.686478726 +0000 UTC m=+5.565841977 container died 25d0e7bbd679ae0542366e97242bdc074b8b0b099520a70236e23660a19c1ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:22:19.984 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:19.645+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:19.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:19.645+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:19.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:19.645+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:19.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:19.645+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:19.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:19.645+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:19.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:19 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:19.645+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:20.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 podman[71015]: 2024-04-25 17:22:20.060105058 +0000 UTC m=+5.939468310 container cleanup 25d0e7bbd679ae0542366e97242bdc074b8b0b099520a70236e23660a19c1ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T17:22:20.281 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 bash[71015]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8 2024-04-25T17:22:20.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:19.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:20.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:19.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:20.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:19.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:20.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:19.991+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:20.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:19.991+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:20.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:19 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:19.991+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:20.635 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 podman[71032]: 2024-04-25 17:22:20.303855133 +0000 UTC m=+0.616936025 container remove 25d0e7bbd679ae0542366e97242bdc074b8b0b099520a70236e23660a19c1ccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T17:22:20.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:20.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:20.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:20.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:20.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:20.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:20.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:20.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:20.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:20 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:20.239+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:20.885 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 podman[71121]: 2024-04-25 17:22:20.537845481 +0000 UTC m=+0.018378441 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:22:20.886 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 podman[71121]: 2024-04-25 17:22:20.637838172 +0000 UTC m=+0.118371130 container create 17d68080c4b2c379cf53cef87de4fb3d642fa5f0eede67b5d7765fdd22b37a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:22:20.886 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 podman[71121]: 2024-04-25 17:22:20.882430196 +0000 UTC m=+0.362963159 container init 17d68080c4b2c379cf53cef87de4fb3d642fa5f0eede67b5d7765fdd22b37a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T17:22:20.889 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 podman[71121]: 2024-04-25 17:22:20.885609984 +0000 UTC m=+0.366142947 container start 17d68080c4b2c379cf53cef87de4fb3d642fa5f0eede67b5d7765fdd22b37a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T17:22:20.984 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:20.672+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:20.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:20.672+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:20.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:20.672+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:20.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:20.672+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:20.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:20.672+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:20.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:20 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:20.672+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:21.175 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:20 smithi171 podman[71121]: 2024-04-25 17:22:20.991304884 +0000 UTC m=+0.471837863 container attach 17d68080c4b2c379cf53cef87de4fb3d642fa5f0eede67b5d7765fdd22b37a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-25T17:22:21.175 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:21 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.032+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:21.175 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:21 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.032+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:21.175 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:21 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.032+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:21.176 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:21 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.032+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:21.176 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:21 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.032+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:21.176 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:21 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.032+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:21.450 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 25 17:22:21 smithi171 podman[71121]: 2024-04-25 17:22:21.175405448 +0000 UTC m=+0.655938414 container died 17d68080c4b2c379cf53cef87de4fb3d642fa5f0eede67b5d7765fdd22b37a6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T17:22:21.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:21.262+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:21.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:21.262+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:21.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:21.262+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:21.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:21.262+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:21.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:21.262+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:21.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:21 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:21.262+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:55.066789+0000 front 2024-04-25T17:21:55.066646+0000 (oldest deadline 2024-04-25T17:22:20.966624+0000) 2024-04-25T17:22:21.764 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T17:22:21.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:21.801 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T17:22:21.801 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T17:22:21.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.9 2024-04-25T17:22:21.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:21.626+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:21.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:21.626+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:21.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:21.626+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:21.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:21.626+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:21.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:21.626+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:21.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:21 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:21.626+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:22.111 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:21 smithi096 systemd[1]: Stopping Ceph osd.9 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:22.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.999+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:22.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.999+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:22.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.999+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:22.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.999+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:22.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.999+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:22.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:21.999+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:22.439 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.110+0000 7fc7bdf6f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.110+0000 7fc7bdf6f700 -1 osd.9 91 *** Got signal Terminated *** 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.110+0000 7fc7bdf6f700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:22.440 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:22 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:22.217+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:55.066789+0000 front 2024-04-25T17:21:55.066646+0000 (oldest deadline 2024-04-25T17:22:20.966624+0000) 2024-04-25T17:22:22.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:22.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:22.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:22.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:22.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:22.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:22.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:22.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:22.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:22.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:22.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:22 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:22.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:23.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:22.977+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:23.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:22.977+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:23.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:22.977+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:23.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:22.977+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:23.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:22.977+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:23.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:22 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:22.977+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:23.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:23 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:23.199+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:23.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:23 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:23.199+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:23.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:23 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:23.199+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:23.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:23 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:23.199+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:23.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:23 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:23.199+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:23.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:23 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:23.199+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:55.066789+0000 front 2024-04-25T17:21:55.066646+0000 (oldest deadline 2024-04-25T17:22:20.966624+0000) 2024-04-25T17:22:23.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:23 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:23.662+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:23.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:23 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:23.662+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:23.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:23 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:23.662+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:23.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:23 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:23.662+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:23.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:23 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:23.662+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:23.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:23 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:23.662+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:24.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:23 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:23.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:24.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:23 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:23.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:24.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:23 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:23.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:24.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:23 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:23.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:24.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:23 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:23.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:24.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:23 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:23.990+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:24.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:24 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:24.198+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:24.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:24 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:24.198+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:24.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:24 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:24.198+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:24.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:24 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:24.198+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:24.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:24 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:24.198+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:24.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:24 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:24.198+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:55.066789+0000 front 2024-04-25T17:21:55.066646+0000 (oldest deadline 2024-04-25T17:22:20.966624+0000) 2024-04-25T17:22:24.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:24 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:24.613+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:24.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:24 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:24.613+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:24.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:24 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:24.613+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:24.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:24 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:24.613+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:24.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:24 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:24.613+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:24.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:24 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:24.613+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:24.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:24 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:24.613+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:25.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:25 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:25.028+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:25.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:25 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:25.028+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:25.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:25 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:25.028+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:25.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:25 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:25.028+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:25.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:25 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:25.028+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:25.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:25 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:25.028+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:25.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:25 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:25.237+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:25.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:25 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:25.237+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:25.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:25 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:25.238+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:25.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:25 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:25.238+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:25.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:25 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:25.238+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:25.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:25 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:25.238+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:55.066789+0000 front 2024-04-25T17:21:55.066646+0000 (oldest deadline 2024-04-25T17:22:20.966624+0000) 2024-04-25T17:22:25.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:25 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:25.575+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:25.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:25 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:25.575+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:25.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:25 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:25.575+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:25.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:25 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:25.575+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:25.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:25 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:25.575+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:25.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:25 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:25.575+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:25.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:25 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:25.575+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:26.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.018+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:26.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.018+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:26.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.018+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:26.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.018+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:26.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.018+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:26.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.018+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:26.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:26.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:12.960394+0000 front 2024-04-25T17:21:12.960348+0000 (oldest deadline 2024-04-25T17:21:38.860489+0000) 2024-04-25T17:22:26.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:26.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.962131+0000 front 2024-04-25T17:21:22.961952+0000 (oldest deadline 2024-04-25T17:21:47.661997+0000) 2024-04-25T17:22:26.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:26.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:31.064000+0000 front 2024-04-25T17:21:31.064014+0000 (oldest deadline 2024-04-25T17:21:55.163793+0000) 2024-04-25T17:22:26.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:26.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:35.164217+0000 front 2024-04-25T17:21:35.164139+0000 (oldest deadline 2024-04-25T17:21:59.864364+0000) 2024-04-25T17:22:26.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:26.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.065963+0000 front 2024-04-25T17:21:45.066055+0000 (oldest deadline 2024-04-25T17:22:09.765654+0000) 2024-04-25T17:22:26.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:26.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:55.066789+0000 front 2024-04-25T17:21:55.066646+0000 (oldest deadline 2024-04-25T17:22:20.966624+0000) 2024-04-25T17:22:26.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:26 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9[64069]: 2024-04-25T17:22:26.274+0000 7fc7b651d700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:22:00.966973+0000 front 2024-04-25T17:22:00.967034+0000 (oldest deadline 2024-04-25T17:22:25.667030+0000) 2024-04-25T17:22:26.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:26 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:26.585+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:26.993 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:26 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:26.585+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:26.993 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:26 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:26.585+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:26.994 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:26 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:26.585+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:26.994 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:26 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:26.585+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:26.994 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:26 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:26.585+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:26.994 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:26 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:26.585+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:27.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.995+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:27.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.995+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:27.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.995+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:27.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.995+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:27.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.995+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:27.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:26 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:26.995+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:27.843 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:27 smithi096 podman[107221]: 2024-04-25 17:22:27.560064566 +0000 UTC m=+5.560856485 container died aae4a54d3bbe65155107147b7c75bd2d2dafefd552bd1ffad1065ccef1afa55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T17:22:27.843 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:27 smithi096 podman[107221]: 2024-04-25 17:22:27.843078949 +0000 UTC m=+5.843870860 container cleanup aae4a54d3bbe65155107147b7c75bd2d2dafefd552bd1ffad1065ccef1afa55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-25T17:22:27.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:27 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:27.572+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:27.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:27 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:27.572+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:27.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:27 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:27.572+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:27.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:27 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:27.572+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:27.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:27 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:27.572+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:27.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:27 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:27.572+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:27.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:27 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:27.572+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:28.189 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:27 smithi096 bash[107221]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9 2024-04-25T17:22:28.189 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:28 smithi096 podman[107239]: 2024-04-25 17:22:28.085772513 +0000 UTC m=+0.525223365 container remove aae4a54d3bbe65155107147b7c75bd2d2dafefd552bd1ffad1065ccef1afa55a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T17:22:28.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:28 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:28.007+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:28.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:28 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:28.007+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:28.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:28 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:28.007+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:28.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:28 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:28.007+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:28.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:28 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:28.007+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:28.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:28 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:28.007+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:28.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:28 smithi096 podman[107329]: 2024-04-25 17:22:28.411624062 +0000 UTC m=+0.114047606 container create 77ab0fcc6d1fc9dd3f0a61d28fb848a19282b72b722a0cf1a6c35f9ea586958c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T17:22:28.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:28 smithi096 podman[107329]: 2024-04-25 17:22:28.315705947 +0000 UTC m=+0.018129504 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:22:28.689 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:28 smithi096 podman[107329]: 2024-04-25 17:22:28.647022417 +0000 UTC m=+0.349445961 container init 77ab0fcc6d1fc9dd3f0a61d28fb848a19282b72b722a0cf1a6c35f9ea586958c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T17:22:28.690 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:28 smithi096 podman[107329]: 2024-04-25 17:22:28.650190981 +0000 UTC m=+0.352614526 container start 77ab0fcc6d1fc9dd3f0a61d28fb848a19282b72b722a0cf1a6c35f9ea586958c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:22:28.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:28.614+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:28.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:28.614+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:28.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:28.614+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:28.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:28.614+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:28.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:28.614+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:28.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:28.614+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:28.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:28 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:28.614+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:29.190 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:28 smithi096 podman[107329]: 2024-04-25 17:22:28.738811113 +0000 UTC m=+0.441234666 container attach 77ab0fcc6d1fc9dd3f0a61d28fb848a19282b72b722a0cf1a6c35f9ea586958c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:22:29.190 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:28 smithi096 podman[107329]: 2024-04-25 17:22:28.935507311 +0000 UTC m=+0.637930856 container died 77ab0fcc6d1fc9dd3f0a61d28fb848a19282b72b722a0cf1a6c35f9ea586958c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:22:29.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:29.002+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:29.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:29.002+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:29.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:29.002+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:29.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:29.002+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:29.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:29.002+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:29.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:29.002+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:29.479 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 17:22:29 smithi096 podman[107355]: 2024-04-25 17:22:29.477427236 +0000 UTC m=+0.536951677 container remove 77ab0fcc6d1fc9dd3f0a61d28fb848a19282b72b722a0cf1a6c35f9ea586958c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:22:29.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T17:22:29.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:29.540 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T17:22:29.540 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T17:22:29.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.10 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 systemd[1]: Stopping Ceph osd.10 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.641+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.862+0000 7f5ca8127700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.862+0000 7f5ca8127700 -1 osd.10 91 *** Got signal Terminated *** 2024-04-25T17:22:29.864 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:29 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:29.862+0000 7f5ca8127700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:22:30.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:30 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:30.041+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:30.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:30 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:30.041+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:30.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:30 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:30.041+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:30.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:30 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:30.041+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:30.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:30 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:30.041+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:30.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:30 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:30.041+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:30.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:30 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:30.655+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:30.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:30 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:30.655+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:30.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:30 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:30.655+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:30.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:30 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:30.655+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:30.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:30 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:30.655+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:30.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:30 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:30.655+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:30.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:30 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:30.655+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:31.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:31 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:31.024+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:31.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:31 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:31.024+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:31.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:31 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:31.024+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:31.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:31 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:31.024+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:31.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:31 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:31.024+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:31.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:31 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:31.024+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:31.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:31 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:31.677+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:31.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:31 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:31.677+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:31.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:31 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:31.677+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:31.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:31 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:31.677+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:31.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:31 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:31.677+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:31.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:31 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:31.677+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:31.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:31 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:31.677+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:32.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:32 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:32.000+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:32.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:32 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:32.000+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:32.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:32 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:32.000+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:32.183 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:32 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:32.000+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:32.184 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:32 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:32.000+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:32.184 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:32 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:32.000+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:32.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:32 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:32.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:32.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:32 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:32.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:32.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:32 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:32.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:32.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:32 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:32.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:32.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:32 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:32.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:32.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:32 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:32.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:32.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:32 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:32.632+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:33.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:33.005+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:33.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:33.005+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:33.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:33.005+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:33.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:33.005+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:33.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:33.005+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:33.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:33.005+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:33.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:33.623+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:33.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:33.623+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:33.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:33.623+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:33.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:33.623+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:33.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:33.623+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:33.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:33.623+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:33.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:33 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:33.623+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:34.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:34 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:34.053+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:34.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:34 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:34.053+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:34.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:34 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:34.053+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:34.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:34 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:34.053+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:34.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:34 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:34.053+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:34.532 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:34 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:34.053+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:34.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:34 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:34.637+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:09.932235+0000 front 2024-04-25T17:21:09.932368+0000 (oldest deadline 2024-04-25T17:21:40.533030+0000) 2024-04-25T17:22:34.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:34 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:34.637+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:20.533449+0000 front 2024-04-25T17:21:20.533425+0000 (oldest deadline 2024-04-25T17:21:45.833611+0000) 2024-04-25T17:22:34.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:34 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:34.637+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:29.935057+0000 front 2024-04-25T17:21:29.934846+0000 (oldest deadline 2024-04-25T17:21:55.234716+0000) 2024-04-25T17:22:34.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:34 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:34.637+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:36.335782+0000 front 2024-04-25T17:21:36.335826+0000 (oldest deadline 2024-04-25T17:22:02.235885+0000) 2024-04-25T17:22:34.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:34 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:34.637+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.136877+0000 front 2024-04-25T17:21:45.136913+0000 (oldest deadline 2024-04-25T17:22:11.036962+0000) 2024-04-25T17:22:34.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:34 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:34.637+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:54.538325+0000 front 2024-04-25T17:21:54.538448+0000 (oldest deadline 2024-04-25T17:22:18.638191+0000) 2024-04-25T17:22:34.986 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:34 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10[57620]: 2024-04-25T17:22:34.637+0000 7f5ca46dd700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T17:21:59.739203+0000 front 2024-04-25T17:21:59.739246+0000 (oldest deadline 2024-04-25T17:22:24.438919+0000) 2024-04-25T17:22:35.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:35 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:35.098+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:35.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:35 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:35.098+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:35.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:35 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:35.098+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:35.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:35 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:35.098+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:35.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:35 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:35.098+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:35.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:35 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:35.098+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:35.697 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:35 smithi097 podman[69428]: 2024-04-25 17:22:35.4064334 +0000 UTC m=+5.671157797 container died 7ae6eb61db113e0b2145feb86caf5de7f8dc11f8902aba00f6341aa9f7a3c748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T17:22:35.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:35 smithi097 podman[69428]: 2024-04-25 17:22:35.796816165 +0000 UTC m=+6.061540555 container cleanup 7ae6eb61db113e0b2145feb86caf5de7f8dc11f8902aba00f6341aa9f7a3c748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T17:22:35.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:35 smithi097 bash[69428]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10 2024-04-25T17:22:36.336 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:36 smithi097 podman[69445]: 2024-04-25 17:22:36.031767142 +0000 UTC m=+0.622957311 container remove 7ae6eb61db113e0b2145feb86caf5de7f8dc11f8902aba00f6341aa9f7a3c748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:22:36.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:36.076+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:36.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:36.076+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:36.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:36.076+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:36.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:36.076+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:36.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:36.076+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:36.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:36.076+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:36.532 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:36 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:36.076+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T17:22:09.692115+0000 front 2024-04-25T17:22:09.692147+0000 (oldest deadline 2024-04-25T17:22:35.591979+0000) 2024-04-25T17:22:36.676 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:36 smithi097 podman[69534]: 2024-04-25 17:22:36.239431776 +0000 UTC m=+0.017924334 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:22:36.677 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:36 smithi097 podman[69534]: 2024-04-25 17:22:36.340859153 +0000 UTC m=+0.119351696 container create 8135f6c2378034eb1d0ce7b4dfdc65eb1b64cf2aede2bcb442a59d78bb5b0d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:22:36.677 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:36 smithi097 podman[69534]: 2024-04-25 17:22:36.576549805 +0000 UTC m=+0.355042349 container init 8135f6c2378034eb1d0ce7b4dfdc65eb1b64cf2aede2bcb442a59d78bb5b0d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:22:36.677 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:36 smithi097 podman[69534]: 2024-04-25 17:22:36.579545685 +0000 UTC m=+0.358038230 container start 8135f6c2378034eb1d0ce7b4dfdc65eb1b64cf2aede2bcb442a59d78bb5b0d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T17:22:36.677 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:36 smithi097 podman[69534]: 2024-04-25 17:22:36.676672919 +0000 UTC m=+0.455165464 container attach 8135f6c2378034eb1d0ce7b4dfdc65eb1b64cf2aede2bcb442a59d78bb5b0d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-04-25T17:22:36.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 17:22:36 smithi097 podman[69534]: 2024-04-25 17:22:36.865804301 +0000 UTC m=+0.644296837 container died 8135f6c2378034eb1d0ce7b4dfdc65eb1b64cf2aede2bcb442a59d78bb5b0d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:22:37.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T17:22:37.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.100+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:37.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.100+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:37.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.100+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:37.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.100+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:37.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.100+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:37.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.100+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:37.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.100+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T17:22:09.692115+0000 front 2024-04-25T17:22:09.692147+0000 (oldest deadline 2024-04-25T17:22:35.591979+0000) 2024-04-25T17:22:37.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:37.553 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T17:22:37.553 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T17:22:37.553 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.11 2024-04-25T17:22:37.866 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 systemd[1]: Stopping Ceph osd.11 for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:38.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.866+0000 7fe1a1bc5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:38.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.866+0000 7fe1a1bc5700 -1 osd.11 91 *** Got signal Terminated *** 2024-04-25T17:22:38.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:37 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:37.866+0000 7fe1a1bc5700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T17:22:38.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:38.112+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:38.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:38.112+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:38.281 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:38.112+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:38.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:38.112+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:38.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:38.112+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:38.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:38.112+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:38.282 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:38 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:38.112+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T17:22:09.692115+0000 front 2024-04-25T17:22:09.692147+0000 (oldest deadline 2024-04-25T17:22:35.591979+0000) 2024-04-25T17:22:39.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:39.080+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:39.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:39.080+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:39.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:39.080+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:39.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:39.080+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:39.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:39.080+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:39.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:39.080+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:39.532 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:39 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:39.080+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T17:22:09.692115+0000 front 2024-04-25T17:22:09.692147+0000 (oldest deadline 2024-04-25T17:22:35.591979+0000) 2024-04-25T17:22:40.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:40.129+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:40.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:40.129+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:40.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:40.129+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:40.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:40.129+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:40.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:40.129+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:40.532 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:40.129+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:40.532 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:40 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:40.129+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T17:22:09.692115+0000 front 2024-04-25T17:22:09.692147+0000 (oldest deadline 2024-04-25T17:22:35.591979+0000) 2024-04-25T17:22:41.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:41.092+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:41.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:41.092+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:41.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:41.092+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:41.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:41.092+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:41.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:41.092+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:41.532 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:41.092+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:41.532 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:41 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:41.092+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T17:22:09.692115+0000 front 2024-04-25T17:22:09.692147+0000 (oldest deadline 2024-04-25T17:22:35.591979+0000) 2024-04-25T17:22:42.486 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T17:21:11.081539+0000 front 2024-04-25T17:21:11.081502+0000 (oldest deadline 2024-04-25T17:21:36.981512+0000) 2024-04-25T17:22:42.486 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T17:21:22.084080+0000 front 2024-04-25T17:21:22.083863+0000 (oldest deadline 2024-04-25T17:21:46.684445+0000) 2024-04-25T17:22:42.486 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-25T17:21:30.185324+0000 front 2024-04-25T17:21:30.185409+0000 (oldest deadline 2024-04-25T17:21:52.485434+0000) 2024-04-25T17:22:42.486 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T17:21:37.186800+0000 front 2024-04-25T17:21:37.186842+0000 (oldest deadline 2024-04-25T17:22:00.086678+0000) 2024-04-25T17:22:42.486 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T17:21:45.788864+0000 front 2024-04-25T17:21:45.788776+0000 (oldest deadline 2024-04-25T17:22:10.389142+0000) 2024-04-25T17:22:42.487 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-25T17:21:51.489996+0000 front 2024-04-25T17:21:51.489939+0000 (oldest deadline 2024-04-25T17:22:17.389979+0000) 2024-04-25T17:22:42.487 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T17:22:09.692115+0000 front 2024-04-25T17:22:09.692147+0000 (oldest deadline 2024-04-25T17:22:35.591979+0000) 2024-04-25T17:22:42.487 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:42 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11[59324]: 2024-04-25T17:22:42.045+0000 7fe19e17b700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-25T17:22:16.693063+0000 front 2024-04-25T17:22:16.693091+0000 (oldest deadline 2024-04-25T17:22:41.393119+0000) 2024-04-25T17:22:43.531 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:43 smithi171 podman[71243]: 2024-04-25 17:22:43.254395623 +0000 UTC m=+5.509055661 container died f5cd755b13b617e9ac54165a006f489688ad9d06a12ce42d323d781b039672ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T17:22:43.843 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:43 smithi171 podman[71243]: 2024-04-25 17:22:43.625047934 +0000 UTC m=+5.879707962 container cleanup f5cd755b13b617e9ac54165a006f489688ad9d06a12ce42d323d781b039672ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T17:22:43.843 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:43 smithi171 bash[71243]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11 2024-04-25T17:22:44.163 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:43 smithi171 podman[71260]: 2024-04-25 17:22:43.843252013 +0000 UTC m=+0.585485807 container remove f5cd755b13b617e9ac54165a006f489688ad9d06a12ce42d323d781b039672ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T17:22:44.415 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:44 smithi171 podman[71353]: 2024-04-25 17:22:44.06666377 +0000 UTC m=+0.018939057 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T17:22:44.415 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:44 smithi171 podman[71353]: 2024-04-25 17:22:44.177916839 +0000 UTC m=+0.130192160 container create 0cec5c056916208e8da8dc5e2d2eec229f3e350f88a8dd88b6b1d2847ea16384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T17:22:44.718 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:44 smithi171 podman[71353]: 2024-04-25 17:22:44.414720004 +0000 UTC m=+0.366995292 container init 0cec5c056916208e8da8dc5e2d2eec229f3e350f88a8dd88b6b1d2847ea16384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T17:22:44.718 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:44 smithi171 podman[71353]: 2024-04-25 17:22:44.418537336 +0000 UTC m=+0.370812621 container start 0cec5c056916208e8da8dc5e2d2eec229f3e350f88a8dd88b6b1d2847ea16384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:22:44.718 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:44 smithi171 podman[71353]: 2024-04-25 17:22:44.489478064 +0000 UTC m=+0.441753346 container attach 0cec5c056916208e8da8dc5e2d2eec229f3e350f88a8dd88b6b1d2847ea16384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T17:22:45.010 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 25 17:22:44 smithi171 podman[71353]: 2024-04-25 17:22:44.718454208 +0000 UTC m=+0.670729488 container died 0cec5c056916208e8da8dc5e2d2eec229f3e350f88a8dd88b6b1d2847ea16384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T17:22:45.280 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T17:22:45.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:45.319 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T17:22:45.319 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T17:22:45.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.a 2024-04-25T17:22:45.604 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:22:45 smithi096 systemd[1]: Stopping Ceph mds.a for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:45.939 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:22:45 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a[70793]: 2024-04-25T17:22:45.602+0000 7fb9b182f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:45.940 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:22:45 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a[70793]: 2024-04-25T17:22:45.602+0000 7fb9b182f700 -1 mds.a *** got signal Terminated *** 2024-04-25T17:22:49.918 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:22:49 smithi096 podman[107449]: 2024-04-25 17:22:49.609430437 +0000 UTC m=+4.099026330 container died 6e4ee577a3d3a806766c849dd857e0c4a62978be9b7c20ef325fd5c9f5cdc7a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:22:49.919 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:22:49 smithi096 podman[107449]: 2024-04-25 17:22:49.918860626 +0000 UTC m=+4.408456518 container cleanup 6e4ee577a3d3a806766c849dd857e0c4a62978be9b7c20ef325fd5c9f5cdc7a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T17:22:50.189 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:22:49 smithi096 bash[107449]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a 2024-04-25T17:22:50.189 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 17:22:50 smithi096 podman[107461]: 2024-04-25 17:22:50.128397559 +0000 UTC m=+0.513283446 container remove 6e4ee577a3d3a806766c849dd857e0c4a62978be9b7c20ef325fd5c9f5cdc7a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T17:22:50.253 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T17:22:50.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:50.287 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T17:22:50.287 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T17:22:50.287 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.d 2024-04-25T17:22:50.597 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:50 smithi096 systemd[1]: Stopping Ceph mds.d for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:50.939 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d[69830]: 2024-04-25T17:22:50.596+0000 7f2164768700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:50.939 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:50 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d[69830]: 2024-04-25T17:22:50.596+0000 7f2164768700 -1 mds.d *** got signal Terminated *** 2024-04-25T17:22:54.689 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:54 smithi096 podman[107570]: 2024-04-25 17:22:54.438069383 +0000 UTC m=+3.939249855 container died eea5ecc06d43160d0c028de4816e1b4a734f31af572c57150c7be5b847fe0c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:22:54.957 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:54 smithi096 podman[107570]: 2024-04-25 17:22:54.780217676 +0000 UTC m=+4.281398149 container cleanup eea5ecc06d43160d0c028de4816e1b4a734f31af572c57150c7be5b847fe0c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:22:54.957 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:54 smithi096 bash[107570]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d 2024-04-25T17:22:55.196 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T17:22:55.215 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:54 smithi096 podman[107584]: 2024-04-25 17:22:54.956518468 +0000 UTC m=+0.519752816 container remove eea5ecc06d43160d0c028de4816e1b4a734f31af572c57150c7be5b847fe0c4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-25T17:22:55.216 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:55 smithi096 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-25T17:22:55.216 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:55 smithi096 systemd[1]: Stopped Ceph mds.d for 1be09d02-0325-11ef-bc93-c7b262605968. 2024-04-25T17:22:55.216 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 17:22:55 smithi096 systemd[1]: ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.d.service: Consumed 1.083s CPU time. 2024-04-25T17:22:55.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:22:55.227 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T17:22:55.228 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T17:22:55.228 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.g 2024-04-25T17:22:55.534 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:22:55 smithi096 systemd[1]: Stopping Ceph mds.g for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:22:55.939 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:22:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g[70427]: 2024-04-25T17:22:55.533+0000 7f53f57c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:22:55.939 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:22:55 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g[70427]: 2024-04-25T17:22:55.533+0000 7f53f57c8700 -1 mds.g *** got signal Terminated *** 2024-04-25T17:22:59.850 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:22:59 smithi096 podman[107690]: 2024-04-25 17:22:59.462656828 +0000 UTC m=+4.023356678 container died 4ab184210c24cf63c80cda7a4c48174e8c8dd5325052cdddbaee0c315485f960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T17:23:00.189 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:22:59 smithi096 podman[107690]: 2024-04-25 17:22:59.924916686 +0000 UTC m=+4.485616538 container cleanup 4ab184210c24cf63c80cda7a4c48174e8c8dd5325052cdddbaee0c315485f960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:23:00.189 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 17:22:59 smithi096 bash[107690]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-g 2024-04-25T17:23:00.361 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T17:23:00.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:00.391 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T17:23:00.392 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T17:23:00.392 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.j 2024-04-25T17:23:00.712 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:23:00 smithi096 systemd[1]: Stopping Ceph mds.j for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:01.189 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:23:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j[69474]: 2024-04-25T17:23:00.712+0000 7fa6cb300700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:01.189 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:23:00 smithi096 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j[69474]: 2024-04-25T17:23:00.712+0000 7fa6cb300700 -1 mds.j *** got signal Terminated *** 2024-04-25T17:23:03.863 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:23:03 smithi096 podman[107809]: 2024-04-25 17:23:03.589545015 +0000 UTC m=+2.982553425 container died f8b3ae12fee44ac8ee224fbc782eb74590486ed4cbb18d1ec9b4c188ff9258a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:23:04.189 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:23:03 smithi096 podman[107809]: 2024-04-25 17:23:03.944842637 +0000 UTC m=+3.337851043 container cleanup f8b3ae12fee44ac8ee224fbc782eb74590486ed4cbb18d1ec9b4c188ff9258a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T17:23:04.189 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 17:23:03 smithi096 bash[107809]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-j 2024-04-25T17:23:04.321 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T17:23:04.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:04.352 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T17:23:04.352 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T17:23:04.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.b 2024-04-25T17:23:04.665 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:23:04 smithi097 systemd[1]: Stopping Ceph mds.b for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:04.985 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:23:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b[62836]: 2024-04-25T17:23:04.663+0000 7f80033a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:04.985 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:23:04 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b[62836]: 2024-04-25T17:23:04.663+0000 7f80033a3700 -1 mds.b *** got signal Terminated *** 2024-04-25T17:23:07.273 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:23:07 smithi097 podman[69640]: 2024-04-25 17:23:07.004206365 +0000 UTC m=+2.460978730 container died df6fe178286b9070a9e8cbad6d66690a985a7285ef3758102527bd372c94d399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T17:23:07.649 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:23:07 smithi097 podman[69640]: 2024-04-25 17:23:07.372383424 +0000 UTC m=+2.829155788 container cleanup df6fe178286b9070a9e8cbad6d66690a985a7285ef3758102527bd372c94d399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T17:23:07.649 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 17:23:07 smithi097 bash[69640]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-b 2024-04-25T17:23:07.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T17:23:07.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:07.841 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T17:23:07.841 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T17:23:07.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.e 2024-04-25T17:23:08.160 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:23:07 smithi097 systemd[1]: Stopping Ceph mds.e for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:08.484 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:23:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e[62210]: 2024-04-25T17:23:08.159+0000 7f9616ba7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:08.485 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:23:08 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e[62210]: 2024-04-25T17:23:08.159+0000 7f9616ba7700 -1 mds.e *** got signal Terminated *** 2024-04-25T17:23:09.611 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:23:09 smithi097 podman[69761]: 2024-04-25 17:23:09.326136573 +0000 UTC m=+1.271420337 container died 5f451bd1b1d4f1e83622efd80814d738efbd9a91213f5975981a16d0d8db0def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T17:23:09.969 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:23:09 smithi097 podman[69761]: 2024-04-25 17:23:09.709956401 +0000 UTC m=+1.655240163 container cleanup 5f451bd1b1d4f1e83622efd80814d738efbd9a91213f5975981a16d0d8db0def (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T17:23:09.970 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 17:23:09 smithi097 bash[69761]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-e 2024-04-25T17:23:10.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T17:23:10.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:10.170 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T17:23:10.170 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T17:23:10.170 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h 2024-04-25T17:23:10.484 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:23:10 smithi097 systemd[1]: Stopping Ceph mds.h for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:10.793 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:23:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[66208]: 2024-04-25T17:23:10.530+0000 7f5e92f49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:10.794 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:23:10 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h[66208]: 2024-04-25T17:23:10.530+0000 7f5e92f49700 -1 mds.h *** got signal Terminated *** 2024-04-25T17:23:14.833 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:23:14 smithi097 podman[69891]: 2024-04-25 17:23:14.545142859 +0000 UTC m=+4.126623246 container died 900c48e9406c92294cfe307f6f4beca06eba7fc111a442c53c70fdeb0e2bce29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T17:23:15.190 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:23:14 smithi097 podman[69891]: 2024-04-25 17:23:14.92277301 +0000 UTC m=+4.504253405 container cleanup 900c48e9406c92294cfe307f6f4beca06eba7fc111a442c53c70fdeb0e2bce29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T17:23:15.190 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 17:23:14 smithi097 bash[69891]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-h 2024-04-25T17:23:15.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T17:23:15.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:15.370 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T17:23:15.370 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T17:23:15.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.k 2024-04-25T17:23:15.727 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:23:15 smithi097 systemd[1]: Stopping Ceph mds.k for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:15.985 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:23:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k[61887]: 2024-04-25T17:23:15.726+0000 7f4e1240c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:15.985 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:23:15 smithi097 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k[61887]: 2024-04-25T17:23:15.726+0000 7f4e1240c700 -1 mds.k *** got signal Terminated *** 2024-04-25T17:23:20.051 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:23:19 smithi097 podman[70012]: 2024-04-25 17:23:19.739472527 +0000 UTC m=+4.158989897 container died 900d491e8a39cc7e923e9ebdb0e509c322579f40f78eae03211edf7fff2f8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T17:23:20.051 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:23:20 smithi097 podman[70012]: 2024-04-25 17:23:20.051123175 +0000 UTC m=+4.470640538 container cleanup 900d491e8a39cc7e923e9ebdb0e509c322579f40f78eae03211edf7fff2f8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T17:23:20.455 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:23:20 smithi097 bash[70012]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k 2024-04-25T17:23:20.455 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 17:23:20 smithi097 podman[70026]: 2024-04-25 17:23:20.235443646 +0000 UTC m=+0.495989973 container remove 900d491e8a39cc7e923e9ebdb0e509c322579f40f78eae03211edf7fff2f8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T17:23:20.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T17:23:20.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:20.511 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T17:23:20.511 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T17:23:20.511 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.c 2024-04-25T17:23:20.811 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:23:20 smithi171 systemd[1]: Stopping Ceph mds.c for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:21.114 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:23:20 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c[64656]: 2024-04-25T17:23:20.810+0000 7f1ae528e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:21.115 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:23:20 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c[64656]: 2024-04-25T17:23:20.810+0000 7f1ae528e700 -1 mds.c *** got signal Terminated *** 2024-04-25T17:23:23.212 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:23:22 smithi171 podman[71460]: 2024-04-25 17:23:22.943911279 +0000 UTC m=+2.239811790 container died 7e8a63c0c96f016865c81866f29714f23e783e7e4470925063c1090836f9c9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T17:23:23.531 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:23:23 smithi171 podman[71460]: 2024-04-25 17:23:23.285858151 +0000 UTC m=+2.581758668 container cleanup 7e8a63c0c96f016865c81866f29714f23e783e7e4470925063c1090836f9c9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:23:23.531 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:23:23 smithi171 bash[71460]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c 2024-04-25T17:23:23.531 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 25 17:23:23 smithi171 podman[71485]: 2024-04-25 17:23:23.454380621 +0000 UTC m=+0.512665048 container remove 7e8a63c0c96f016865c81866f29714f23e783e7e4470925063c1090836f9c9de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T17:23:23.704 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T17:23:23.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:23.739 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T17:23:23.739 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T17:23:23.739 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.f 2024-04-25T17:23:24.031 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 17:23:23 smithi171 systemd[1]: Stopping Ceph mds.f for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:24.531 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 17:23:24 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-f[64347]: 2024-04-25T17:23:24.057+0000 7f16f4d7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:24.531 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 17:23:24 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-f[64347]: 2024-04-25T17:23:24.057+0000 7f16f4d7f700 -1 mds.f *** got signal Terminated *** 2024-04-25T17:23:28.359 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 17:23:28 smithi171 podman[71590]: 2024-04-25 17:23:28.0638211 +0000 UTC m=+4.114261213 container died 7d52ab227ea5c5374c78c92221cda73f207b0d2bc1ebc117d2be5b11f7ae0ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T17:23:28.693 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 17:23:28 smithi171 podman[71590]: 2024-04-25 17:23:28.432894927 +0000 UTC m=+4.483335038 container cleanup 7d52ab227ea5c5374c78c92221cda73f207b0d2bc1ebc117d2be5b11f7ae0ae7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T17:23:28.693 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 25 17:23:28 smithi171 bash[71590]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-f 2024-04-25T17:23:28.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T17:23:28.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:28.884 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T17:23:28.884 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T17:23:28.884 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.i 2024-04-25T17:23:29.204 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:23:28 smithi171 systemd[1]: Stopping Ceph mds.i for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:29.531 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:23:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i[64963]: 2024-04-25T17:23:29.204+0000 7fdf0d1ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:29.531 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:23:29 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i[64963]: 2024-04-25T17:23:29.204+0000 7fdf0d1ae700 -1 mds.i *** got signal Terminated *** 2024-04-25T17:23:32.905 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:23:32 smithi171 podman[71711]: 2024-04-25 17:23:32.638073309 +0000 UTC m=+3.542870962 container died 4372dd765e2bf792e57bfa7089efcd80bbeb8c33f1c1b5a792179926758712c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T17:23:33.281 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:23:32 smithi171 podman[71711]: 2024-04-25 17:23:32.99626635 +0000 UTC m=+3.901064000 container cleanup 4372dd765e2bf792e57bfa7089efcd80bbeb8c33f1c1b5a792179926758712c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T17:23:33.281 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 25 17:23:32 smithi171 bash[71711]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-i 2024-04-25T17:23:33.475 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T17:23:33.505 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:33.505 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T17:23:33.505 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T17:23:33.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.l 2024-04-25T17:23:33.834 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:23:33 smithi171 systemd[1]: Stopping Ceph mds.l for 1be09d02-0325-11ef-bc93-c7b262605968... 2024-04-25T17:23:34.281 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:23:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l[64040]: 2024-04-25T17:23:33.833+0000 7f943991d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T17:23:34.281 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:23:33 smithi171 ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l[64040]: 2024-04-25T17:23:33.833+0000 7f943991d700 -1 mds.l *** got signal Terminated *** 2024-04-25T17:23:37.927 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:23:37 smithi171 podman[71831]: 2024-04-25 17:23:37.647462261 +0000 UTC m=+3.924006419 container died fac5ba1796bceced292879c4ceb4cc2580ba3ce5f1f8dd0ef495de0b686507fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T17:23:38.269 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:23:38 smithi171 podman[71831]: 2024-04-25 17:23:38.009639151 +0000 UTC m=+4.286183310 container cleanup fac5ba1796bceced292879c4ceb4cc2580ba3ce5f1f8dd0ef495de0b686507fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T17:23:38.269 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 25 17:23:38 smithi171 bash[71831]: ceph-1be09d02-0325-11ef-bc93-c7b262605968-mds-l 2024-04-25T17:23:38.388 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1be09d02-0325-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T17:23:38.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T17:23:38.419 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T17:23:38.420 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1be09d02-0325-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T17:23:38.611 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:23:41.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1be09d02-0325-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T17:23:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:23:43.828 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1be09d02-0325-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T17:23:44.016 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:23:46.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:23:46.347 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:23:46.379 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T17:23:46.404 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T17:23:46.406 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi096/crash 2024-04-25T17:23:46.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash -- . 2024-04-25T17:23:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T17:23:46.446 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-04-25T17:23:46.448 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi097/crash 2024-04-25T17:23:46.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash -- . 2024-04-25T17:23:46.494 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T17:23:46.494 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-04-25T17:23:46.496 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi171/crash 2024-04-25T17:23:46.497 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash -- . 2024-04-25T17:23:46.528 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/1be09d02-0325-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T17:23:46.528 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-04-25T17:23:46.529 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T17:23:46.529 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T17:23:46.565 INFO:tasks.cephadm:Compressing logs... 2024-04-25T17:23:46.565 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T17:23:46.607 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T17:23:46.610 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T17:23:46.631 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T17:23:46.631 INFO:teuthology.orchestra.run.smithi096.stderr:: No such file or directory 2024-04-25T17:23:46.631 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log 2024-04-25T17:23:46.631 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T17:23:46.631 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T17:23:46.632 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T17:23:46.632 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T17:23:46.633 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.4% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr: 90.3% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr: 94.7%/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log: -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr: 84.3% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T17:23:46.648 INFO:teuthology.orchestra.run.smithi096.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T17:23:46.649 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log: 94.6% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T17:23:46.649 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:: No such file or directory 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose --/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T17:23:46.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T17:23:46.651 INFO:teuthology.orchestra.run.smithi097.stderr: 91.9% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T17:23:46.654 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr: 87.4% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.5% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr: 94.6% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.11.log: 94.6% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr: 94.6% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.l.log: 92.0% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T17:23:46.655 INFO:teuthology.orchestra.run.smithi171.stderr: 94.7% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T17:23:46.656 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T17:23:46.658 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T17:23:46.659 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T17:23:46.659 INFO:teuthology.orchestra.run.smithi096.stderr: 94.5% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T17:23:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T17:23:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T17:23:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T17:23:46.660 INFO:teuthology.orchestra.run.smithi097.stderr: 84.1% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T17:23:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T17:23:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T17:23:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T17:23:46.667 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T17:23:46.673 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T17:23:46.864 INFO:teuthology.orchestra.run.smithi096.stderr: 90.0% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T17:23:46.878 INFO:teuthology.orchestra.run.smithi171.stderr: 92.0% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T17:23:47.024 INFO:teuthology.orchestra.run.smithi097.stderr: 91.9% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T17:23:47.751 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T17:23:48.895 INFO:teuthology.orchestra.run.smithi097.stderr: 95.9% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T17:23:52.417 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T17:23:52.987 INFO:teuthology.orchestra.run.smithi096.stderr: 92.1% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T17:23:53.560 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T17:23:53.867 INFO:teuthology.orchestra.run.smithi171.stderr: 91.9% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T17:23:54.399 INFO:teuthology.orchestra.run.smithi171.stderr: 91.5% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T17:23:56.003 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T17:23:56.005 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-04-25T17:23:56.006 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m9.384s 2024-04-25T17:23:56.006 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m28.760s 2024-04-25T17:23:56.006 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m1.023s 2024-04-25T17:23:56.523 INFO:teuthology.orchestra.run.smithi171.stderr: 91.7% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T17:23:57.161 INFO:teuthology.orchestra.run.smithi097.stderr: 91.9% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T17:23:57.179 INFO:teuthology.orchestra.run.smithi097.stderr: 91.6% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T17:23:57.251 INFO:teuthology.orchestra.run.smithi097.stderr: 91.7% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T17:23:58.473 INFO:teuthology.orchestra.run.smithi097.stderr: 91.8% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T17:24:00.429 INFO:teuthology.orchestra.run.smithi171.stderr: 91.1% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T17:24:00.431 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-04-25T17:24:00.431 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m13.807s 2024-04-25T17:24:00.431 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m37.662s 2024-04-25T17:24:00.431 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m1.251s 2024-04-25T17:24:15.813 INFO:teuthology.orchestra.run.smithi097.stderr: 93.3% -- replaced with /var/log/ceph/1be09d02-0325-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T17:24:15.815 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T17:24:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m29.191s 2024-04-25T17:24:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:user 1m12.900s 2024-04-25T17:24:15.815 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m2.328s 2024-04-25T17:24:15.815 INFO:tasks.cephadm:Archiving logs... 2024-04-25T17:24:15.816 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi096/log 2024-04-25T17:24:15.817 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T17:24:20.811 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi097/log 2024-04-25T17:24:20.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T17:24:31.453 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi171/log 2024-04-25T17:24:31.454 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T17:24:38.374 INFO:tasks.cephadm:Removing cluster... 2024-04-25T17:24:38.375 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1be09d02-0325-11ef-bc93-c7b262605968 --force 2024-04-25T17:24:38.567 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:24:38.942 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1be09d02-0325-11ef-bc93-c7b262605968 --force 2024-04-25T17:24:39.133 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:24:39.482 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1be09d02-0325-11ef-bc93-c7b262605968 --force 2024-04-25T17:24:39.676 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 1be09d02-0325-11ef-bc93-c7b262605968 2024-04-25T17:24:40.052 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T17:24:40.053 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T17:24:40.071 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T17:24:40.088 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T17:24:40.103 INFO:tasks.cephadm:Teardown complete 2024-04-25T17:24:40.104 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T17:24:40.114 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T17:24:40.115 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T17:24:40.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T17:24:40.131 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T17:24:40.240 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T17:24:40.251 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T17:24:40.251 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T17:24:40.254 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T17:24:40.256 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T17:24:40.268 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-04-25T17:24:40.269 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-25T17:24:40.270 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-25T17:24:40.272 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T17:24:40.272 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-25T17:24:40.272 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 7 377 20 +674us[ +666us] +/- 66ms 2024-04-25T17:24:40.273 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 19 -1471us[-1471us] +/- 72ms 2024-04-25T17:24:40.273 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +702us[ +702us] +/- 69ms 2024-04-25T17:24:40.273 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T17:24:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T17:24:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-25T17:24:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +702us[ +698us] +/- 66ms 2024-04-25T17:24:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -1450us[-1454us] +/- 72ms 2024-04-25T17:24:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +766us[ +762us] +/- 69ms 2024-04-25T17:24:40.274 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T17:24:40.275 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T17:24:40.275 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-25T17:24:40.275 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 +709us[ +701us] +/- 66ms 2024-04-25T17:24:40.275 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 -1456us[-1456us] +/- 72ms 2024-04-25T17:24:40.275 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +766us[ +766us] +/- 69ms 2024-04-25T17:24:40.275 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T17:24:40.276 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T17:24:40.286 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T17:24:40.286 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T17:24:40.355 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T17:24:40.432 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T17:24:40.500 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T17:24:40.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T17:24:40.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:24:40.594 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T17:24:40.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:24:40.622 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T17:24:40.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:24:40.650 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T17:24:40.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T17:24:40.689 INFO:teuthology.task.internal:Duration was 1881.451982 seconds 2024-04-25T17:24:40.689 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T17:24:40.729 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T17:24:40.730 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T17:24:40.732 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T17:24:40.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T17:24:40.769 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T17:24:40.770 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T17:24:40.771 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T17:24:41.276 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T17:24:41.276 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T17:24:41.276 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T17:24:41.297 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T17:24:41.297 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T17:24:41.343 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-04-25T17:24:41.343 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T17:24:41.373 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T17:24:41.373 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T17:24:41.376 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T17:24:41.385 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T17:24:41.438 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T17:24:41.438 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T17:24:41.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T17:24:41.451 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T17:24:41.798 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T17:24:41.809 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T17:24:41.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T17:24:41.834 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T17:24:41.858 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T17:24:41.882 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T17:24:41.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:24:41.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:24:41.900 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:24:41.916 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-04-25T17:24:41.919 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-04-25T17:24:41.944 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-04-25T17:24:41.974 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:24:41.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:24:41.988 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:24:42.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:24:42.011 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:24:42.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:24:42.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T17:24:42.042 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T17:24:42.043 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi096 2024-04-25T17:24:42.043 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T17:24:42.084 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi097 2024-04-25T17:24:42.085 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T17:24:42.121 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673283/remote/smithi171 2024-04-25T17:24:42.121 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T17:24:42.158 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T17:24:42.158 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T17:24:42.162 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T17:24:42.165 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T17:24:42.207 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T17:24:42.432 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T17:24:42.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T17:24:42.649 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T17:24:42.650 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T17:24:42.653 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T17:24:42.655 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T17:24:42.666 INFO:teuthology.orchestra.run.smithi096.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 17:24 /home/ubuntu/cephtest 2024-04-25T17:24:42.668 INFO:teuthology.orchestra.run.smithi097.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 17:24 /home/ubuntu/cephtest 2024-04-25T17:24:42.670 INFO:teuthology.orchestra.run.smithi171.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 17:24 /home/ubuntu/cephtest 2024-04-25T17:24:42.671 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T17:24:42.683 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T17:24:42.751 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/fsx}} duration: 1881.4519817829132 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T17:24:42.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T17:24:42.838 INFO:teuthology.run:pass